Image Retrieval

---

Model Evaluation COSFIRE Filters Approach

Introduction

We obtain the 26 statistically significant sets of hyperparameters from the classification paper along with their respective training, validation, and test descriptors. Based on these descriptors, we perform image hashing for each set of descriptors using a selected set of MLP hyperparameters (for the grid search).

This is the init_notebook_mode cell from ITables v2.0.1
(you should not see this message - is your notebook trusted?)

The outputs below represent an example of a single experiment based on an array of hyperparameters considered to determine the optimal configuration of the MLP hashing architecture (For bit size 32).

Model structure

class CosfireNet(nn.Module):
    def __init__(self, input_size, bitsize, l1_reg, l2_reg):
        super(CosfireNet, self).__init__()
        self.l1_reg = l1_reg
        self.l2_reg = l2_reg
        self.hd = nn.Sequential(
            nn.Linear(input_size, 300),
            nn.BatchNorm1d(300),
            nn.Tanh(),
            nn.Linear(300, 200),
            nn.BatchNorm1d(200),
            nn.Tanh(),
            nn.Linear(200, bitsize),
            nn.Tanh()
        )
    def forward(self, x):
        regularization_loss = 0.0
        for param in self.hd.parameters():
            regularization_loss += torch.sum(torch.abs(param)) * self.l1_reg  # L1 regularization
            regularization_loss += torch.sum(param ** 2) * self.l2_reg  # L2 regularization
        return self.hd(x), regularization_loss

Density plots

mAP Curves

T-SNE_projection plots

Loss curves

Without regularization

With regularization

Tip

tiply, each row represents a unique combination of MLP hyperparameters, and every column represents the results yielded by each of the 26 statistically significant sets of descriptors.

Bit size 32

The results in this presentation are from two experimental designs:

Experiment 1 Results

The thresholding is based on fixed values between -1 and 1 on a step size of 0.1.

Validation Data mAP Results Preview:

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_abs_values	mAP_valid_abs_values_2	mAP_valid_abs_values_3	mAP_valid_abs_values_4	mAP_valid_abs_values_5	mAP_valid_abs_values_6	mAP_valid_abs_values_7	mAP_valid_abs_values_8	mAP_valid_abs_values_9	mAP_valid_abs_values_10	mAP_valid_abs_values_11	mAP_valid_abs_values_12	mAP_valid_abs_values_13	mAP_valid_abs_values_14	mAP_valid_abs_values_15	mAP_valid_abs_values_16	mAP_valid_abs_values_17	mAP_valid_abs_values_18	mAP_valid_abs_values_19	mAP_valid_abs_values_20	mAP_valid_abs_values_21	mAP_valid_abs_values_22	mAP_valid_abs_values_23	mAP_valid_abs_values_25	mAP_valid_abs_values_26
32	372	32	0.1	16	0.00100	24	0.000000e+00	0.000000e+00	84.82	84.18	86.02	87.69	86.39	84.73	84.65	84.76	86.96	85.58	88.88	86.20	87.22	86.42	86.48	86.12	88.42	87.77	87.82	87.82	86.04	88.28	89.25	88.18	88.50
35	372	32	0.1	16	0.00100	24	0.000000e+00	1.000000e-08	84.90	85.24	86.55	85.68	85.95	86.51	86.51	86.33	87.11	85.56	91.15	87.50	85.41	84.28	87.03	87.76	88.19	43.72	88.81	87.18	89.50	87.07	86.97	89.97	88.74
44	372	32	0.1	16	0.00100	24	1.000000e-08	0.000000e+00	86.76	85.87	85.50	85.82	83.94	86.14	86.52	89.03	85.61	85.36	87.82	90.21	85.75	85.88	87.95	86.97	86.65	87.33	88.36	88.13	90.42	89.06	89.74	87.95	89.29
47	372	32	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	85.97	86.44	85.33	86.07	84.88	90.44	84.60	86.48	86.09	85.96	88.27	85.81	85.60	86.64	87.72	86.41	87.40	87.95	91.31	86.88	88.47	88.21	85.73	88.74	87.80
96	372	32	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	85.16	87.37	85.56	85.29	86.22	87.62	43.72	88.58	86.88	87.46	89.44	89.36	86.51	89.21	88.57	43.72	86.57	86.87	89.81	88.68	89.04	89.26	89.90	87.64	87.94
99	372	32	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	86.47	87.08	84.36	86.55	86.57	88.08	87.45	86.88	87.08	86.79	89.07	88.21	86.33	85.30	88.03	86.91	86.80	87.80	89.13	89.74	90.60	89.02	89.72	88.91	89.06
108	372	32	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	87.26	86.15	86.98	86.80	87.01	87.63	85.50	86.17	89.16	88.01	88.29	88.50	89.30	86.56	86.96	86.63	86.66	88.81	88.92	88.35	88.92	87.39	87.13	89.87	90.30
111	372	32	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	84.54	84.88	87.87	86.70	88.55	87.58	87.46	86.24	89.87	88.57	88.20	89.83	89.44	87.26	87.25	86.37	87.98	89.71	87.20	87.60	90.87	86.13	88.69	88.61	89.97
160	372	32	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	86.56	86.59	84.36	85.05	85.06	87.18	84.31	86.53	86.10	85.81	87.21	86.98	86.43	88.40	85.53	79.54	86.77	83.67	88.46	90.74	87.18	88.19	85.79	88.13	88.95
163	372	32	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	85.67	86.79	85.62	85.62	78.36	86.72	85.32	87.70	88.29	87.67	87.96	88.81	87.34	87.92	87.65	85.05	86.15	86.02	89.07	87.85	90.72	87.21	87.75	88.53	90.09
172	372	32	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	85.62	85.72	85.73	86.86	87.20	88.11	87.12	43.72	87.47	87.40	86.28	85.85	86.90	43.72	84.75	88.26	86.92	86.24	89.14	87.61	86.79	87.68	43.72	89.47	86.80
175	372	32	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	83.57	88.60	85.05	85.52	86.78	87.13	85.17	88.53	88.17	86.04	88.37	89.63	89.48	87.61	85.61	88.29	86.31	85.96	88.70	88.63	89.02	88.31	88.51	87.98	88.48
224	372	32	0.1	16	0.00100	36	0.000000e+00	0.000000e+00	83.36	84.48	86.66	85.32	83.12	87.95	84.12	86.79	85.56	87.03	88.05	88.96	85.31	87.24	86.86	87.64	87.18	86.00	88.02	88.37	88.45	88.11	88.62	87.30	88.42
227	372	32	0.1	16	0.00100	36	0.000000e+00	1.000000e-08	86.99	85.76	86.61	87.65	84.57	88.23	86.23	86.47	85.24	85.36	87.21	87.18	86.06	86.31	90.00	87.54	85.60	86.79	89.20	87.90	86.25	86.35	87.34	87.82	87.58
236	372	32	0.1	16	0.00100	36	1.000000e-08	0.000000e+00	87.58	85.74	85.88	84.22	86.54	86.54	84.84	86.50	87.28	84.06	88.49	86.94	85.86	88.56	85.99	85.66	85.20	86.38	88.42	88.66	90.69	89.64	88.39	89.91	88.02
239	372	32	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	85.45	85.46	85.71	85.32	87.23	86.94	83.33	84.72	83.99	86.25	87.65	91.67	86.32	86.19	86.33	87.86	86.93	86.53	88.48	87.75	89.61	87.32	88.75	89.23	88.37
288	372	32	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	86.08	84.04	84.64	86.56	85.57	85.51	86.33	88.45	88.87	86.04	86.50	88.38	85.64	87.41	85.36	86.57	43.72	84.89	43.72	86.61	89.37	86.93	88.91	88.72	88.58
291	372	32	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	85.00	86.97	86.65	84.47	85.31	85.82	84.76	85.96	87.66	86.00	88.10	87.31	89.02	86.31	87.95	87.97	87.17	85.71	86.50	87.66	88.93	88.68	88.41	89.42	88.33
300	372	32	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	86.72	86.78	86.36	83.74	86.64	86.59	86.55	89.40	85.25	85.59	87.28	87.35	88.22	84.84	86.27	87.74	87.33	86.02	89.33	88.86	88.11	88.47	43.72	87.66	88.78
303	372	32	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	85.54	86.12	86.83	83.76	86.98	88.87	86.10	87.55	86.42	87.73	88.23	87.69	87.35	87.17	87.09	86.53	89.32	87.56	88.39	87.90	88.17	88.37	89.39	89.73	88.95
352	372	32	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	86.03	83.61	85.70	84.49	86.15	86.73	81.77	87.36	84.25	87.67	87.49	88.12	87.50	88.50	87.18	87.63	86.02	87.43	86.91	88.82	90.44	87.79	87.34	88.32	88.13
355	372	32	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	85.13	84.65	85.42	84.80	86.61	86.87	87.04	86.11	87.08	87.83	86.10	87.46	86.46	85.79	85.52	87.59	86.48	88.20	87.28	88.50	88.31	87.58	89.13	87.14	88.39
364	372	32	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	85.80	88.79	86.03	85.09	87.04	86.27	86.72	88.49	86.59	88.62	87.48	87.42	85.72	86.16	89.33	86.84	88.79	87.24	88.92	88.41	89.00	90.10	88.34	81.52	88.05
367	372	32	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	86.33	86.86	85.78	89.31	86.21	86.75	85.48	87.72	88.56	86.51	88.42	86.85	88.53	85.72	86.97	87.22	86.84	87.01	89.00	87.50	90.23	88.81	87.17	88.96	87.41
416	372	32	0.1	16	0.00100	48	0.000000e+00	0.000000e+00	85.49	85.61	84.93	83.02	85.13	86.70	85.27	86.69	85.87	88.67	85.30	85.72	88.21	88.16	85.26	87.84	88.21	87.60	87.98	91.87	87.00	88.63	83.66	87.36	89.35
419	372	32	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	85.68	84.95	85.68	85.89	84.71	87.03	85.49	87.81	86.72	86.87	87.43	86.22	87.80	86.80	89.21	88.01	86.27	87.33	88.55	87.40	89.01	88.94	90.13	87.91	90.18
428	372	32	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	86.25	89.25	85.15	85.93	86.30	88.74	85.50	86.49	84.94	88.25	88.57	86.75	87.91	87.10	87.20	87.53	87.98	86.24	89.57	88.24	89.71	89.51	89.01	89.81	86.96
431	372	32	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	85.72	85.81	86.24	87.63	84.94	87.47	84.27	85.81	85.83	87.51	88.27	89.92	87.52	85.80	86.03	86.76	90.59	85.74	89.70	87.99	90.48	88.20	43.72	87.94	89.55
480	372	32	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	86.47	85.23	87.09	86.92	86.11	86.22	84.78	86.88	86.61	87.75	89.84	89.06	86.93	85.81	87.06	87.97	85.70	88.13	89.07	90.33	87.81	92.61	89.41	88.31	89.96
483	372	32	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	85.50	86.98	87.25	87.08	87.69	90.44	83.91	86.28	43.72	87.61	89.66	89.77	86.61	87.63	88.08	87.40	85.53	87.68	90.02	88.12	90.87	88.35	88.06	87.03	89.98
492	372	32	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	84.68	87.15	85.57	86.77	84.22	84.87	85.11	84.73	86.45	85.94	87.13	90.41	86.84	87.28	86.95	87.59	88.94	88.76	88.10	88.11	89.78	88.54	89.73	88.95	88.92
495	372	32	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	85.53	86.15	85.32	86.37	83.24	89.48	90.21	88.14	87.93	88.02	86.47	89.64	90.62	86.92	91.48	87.75	87.88	87.74	88.88	91.25	89.24	88.83	90.97	87.81	89.87
544	372	32	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	85.76	84.85	84.82	85.11	85.59	87.20	83.61	85.53	86.67	86.07	86.43	86.58	88.60	85.22	86.17	88.42	83.21	88.30	92.44	89.44	89.65	90.33	86.70	88.24	88.53
547	372	32	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	86.14	87.67	86.50	87.07	87.51	89.98	88.49	87.38	87.17	89.53	88.39	89.59	88.90	88.02	86.27	90.24	85.71	85.20	92.18	90.23	88.57	88.39	89.98	91.55	91.02
556	372	32	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	86.12	85.43	85.71	84.21	87.99	87.76	87.30	86.55	85.79	91.00	88.08	91.82	87.31	87.38	87.00	87.10	85.69	85.60	91.76	87.93	87.94	89.74	87.72	88.83	87.69
559	372	32	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	86.31	84.13	87.22	86.30	84.36	87.74	84.61	88.01	84.98	83.68	89.35	87.28	87.40	85.55	89.60	87.13	85.28	86.15	89.74	90.44	89.05	89.19	88.67	89.10	89.08
608	372	32	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	83.56	85.05	84.51	84.14	83.06	84.78	43.72	88.35	84.77	86.44	86.46	87.10	87.14	86.39	88.34	86.03	88.50	84.93	87.14	87.30	87.56	89.50	88.01	87.96	88.57
611	372	32	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	86.30	83.19	87.16	85.69	86.07	87.37	84.27	86.98	86.55	85.25	88.60	88.79	88.60	86.71	85.79	86.28	85.25	86.80	87.61	87.08	88.66	88.52	88.66	87.06	86.33
620	372	32	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	85.41	86.57	89.00	86.02	86.98	86.27	84.91	89.05	86.48	84.41	88.09	87.51	86.30	88.63	87.98	89.22	89.58	85.59	87.09	87.54	88.43	88.62	88.15	87.59	88.30
623	372	32	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	88.05	89.92	87.67	85.70	84.82	89.30	85.10	87.01	84.68	84.75	87.70	87.86	87.95	86.42	87.63	86.06	87.76	86.57	89.51	88.64	87.99	87.60	89.93	88.38	89.91
672	372	32	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	84.19	86.51	84.81	83.88	83.99	86.38	84.72	88.25	86.49	87.05	87.72	86.75	85.36	84.18	85.58	84.68	86.42	87.61	87.87	87.91	86.93	88.39	87.59	87.68	88.54
675	372	32	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	85.58	88.65	86.20	84.19	86.13	85.73	88.32	86.90	85.61	87.57	87.88	87.32	87.60	84.88	89.33	86.09	87.18	86.15	88.50	88.65	90.27	89.16	88.68	88.64	88.11
684	372	32	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	86.92	89.19	87.23	84.40	88.17	87.19	85.58	87.34	88.59	87.05	87.24	87.37	86.22	87.59	86.62	87.73	86.14	86.17	86.36	85.31	92.94	88.86	89.37	88.20	89.56
687	372	32	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	86.32	86.76	87.05	87.52	86.74	87.02	86.38	87.31	88.17	86.68	88.64	89.54	87.36	87.10	87.99	88.63	86.43	87.45	89.60	88.46	89.09	89.80	90.05	88.30	88.81
736	372	32	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	85.45	53.61	86.22	87.30	82.06	88.01	84.20	86.60	87.63	88.56	86.51	87.82	87.78	89.33	85.02	88.07	84.12	87.17	86.61	89.09	88.40	90.18	89.80	87.89	88.09
739	372	32	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	86.19	86.57	85.85	88.08	43.72	87.68	86.76	86.56	88.23	87.70	87.34	87.72	86.60	87.38	85.76	86.87	84.97	88.22	85.92	88.71	89.70	88.69	88.98	87.98	89.65
748	372	32	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	86.27	86.75	83.40	87.06	85.29	86.34	85.07	85.44	84.63	86.51	87.03	88.49	85.14	85.94	87.76	88.26	86.22	86.30	88.27	89.87	87.24	89.37	88.79	89.85	87.22
751	372	32	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	86.03	87.19	85.04	87.37	85.30	86.19	87.39	86.97	87.99	85.46	88.67	87.80	87.65	89.27	87.83	86.93	88.18	85.63	87.36	89.86	89.65	88.81	88.68	89.74	87.55
800	372	32	0.1	16	0.00010	36	0.000000e+00	0.000000e+00	87.08	86.50	85.41	89.72	88.55	85.22	84.77	84.99	84.52	86.69	85.35	88.74	87.60	87.41	87.10	87.14	85.50	87.37	90.44	87.82	88.45	88.60	88.80	87.08	87.95
803	372	32	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	85.15	86.45	85.77	85.74	86.61	87.55	85.42	85.27	85.34	85.40	87.80	88.06	87.54	84.11	87.05	86.91	86.82	88.70	88.07	89.00	89.24	86.55	88.26	87.78	88.21
812	372	32	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	85.39	87.11	85.75	89.81	85.89	86.89	83.76	87.52	87.41	87.22	87.48	88.49	85.35	87.29	85.68	86.41	86.29	86.63	88.06	88.86	88.54	86.23	88.62	89.32	86.80
815	372	32	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	84.01	86.70	85.95	87.82	85.53	85.72	84.76	85.41	86.17	86.83	87.74	88.71	87.41	87.31	86.24	87.65	84.51	86.96	88.11	88.02	88.80	87.22	87.94	87.21	88.65
864	372	32	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	84.90	86.55	85.98	85.41	86.51	86.91	85.27	86.66	43.72	87.02	87.74	89.56	86.32	87.07	86.49	87.31	86.40	85.96	88.29	88.42	88.61	88.68	85.50	88.43	89.74
867	372	32	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	86.71	87.05	87.08	87.49	86.33	85.87	85.79	86.08	87.42	84.99	90.08	88.88	85.93	87.92	87.34	86.17	88.05	87.61	89.45	86.30	89.21	89.76	90.59	89.37	87.00
876	372	32	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	87.85	90.38	86.35	85.61	85.52	87.08	85.07	86.97	87.82	85.10	87.25	86.74	85.98	87.29	87.72	89.51	85.87	86.29	86.73	89.50	90.30	86.97	89.55	88.77	87.39
879	372	32	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	87.75	86.42	86.67	87.02	86.32	87.23	86.10	86.96	85.99	85.60	86.49	87.80	86.22	85.90	86.04	86.00	87.17	89.02	88.92	89.02	87.99	88.42	88.08	88.83	88.26
928	372	32	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	84.91	89.98	87.44	85.05	86.76	86.81	82.80	86.31	87.31	89.14	87.40	87.95	85.85	89.82	87.99	86.35	86.13	85.95	86.73	87.35	88.64	87.00	88.95	88.58	83.37
931	372	32	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	84.93	86.73	85.21	84.11	86.37	87.94	85.40	84.53	87.10	86.49	88.67	88.15	84.47	85.86	90.21	88.21	86.53	87.92	86.36	89.30	88.72	90.17	89.64	65.14	89.20
940	372	32	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	83.55	87.05	87.36	86.97	85.60	87.40	85.77	86.47	84.64	84.71	88.59	87.21	88.66	87.75	86.34	87.09	85.03	86.25	87.80	87.85	87.37	88.31	87.77	89.36	88.92
943	372	32	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	86.20	87.80	84.36	87.17	86.24	86.95	85.57	87.44	87.62	85.95	87.03	86.30	87.00	86.59	87.76	86.02	86.43	88.42	88.16	88.62	88.42	88.61	89.49	81.57	88.39
992	372	32	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	84.01	87.10	85.62	83.28	85.06	85.30	85.71	85.78	89.63	86.08	89.19	88.54	87.79	87.86	84.05	87.69	88.92	86.96	89.05	88.98	89.90	89.16	89.58	89.11	83.04
995	372	32	0.1	16	0.00010	48	0.000000e+00	1.000000e-08	86.45	84.83	85.58	85.87	88.63	85.66	86.04	86.45	86.53	85.34	87.39	90.10	86.45	88.39	88.35	86.48	87.80	85.96	87.11	88.65	88.77	85.76	87.98	89.25	87.92
1004	372	32	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	83.95	85.29	89.25	86.02	86.10	86.55	85.95	86.47	85.99	87.79	89.09	84.37	84.57	87.21	87.41	89.61	86.70	86.98	87.60	88.55	88.69	88.95	88.09	88.06	89.35
1007	372	32	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	85.85	85.54	86.20	84.69	90.07	88.63	84.52	88.73	85.73	85.04	87.24	88.04	88.20	87.47	88.51	87.74	89.30	85.58	86.75	87.91	90.04	89.65	88.57	89.55	89.47
1056	372	32	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	86.44	91.60	85.72	84.35	84.12	86.49	85.26	87.74	87.29	84.82	88.83	87.77	87.23	87.91	86.58	87.73	85.50	86.50	89.39	87.08	91.02	89.63	88.15	90.16	89.91
1059	372	32	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	86.70	85.32	87.35	86.52	86.28	87.35	87.73	86.71	86.79	87.70	89.10	88.22	86.50	86.82	87.90	88.35	87.90	87.53	90.55	88.27	87.58	89.38	90.06	89.91	89.77
1068	372	32	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	87.27	87.65	87.08	85.49	87.92	86.72	86.20	87.75	88.60	89.02	88.44	88.49	89.48	85.41	87.67	88.37	87.00	87.65	89.04	89.17	88.42	89.17	88.13	90.03	90.25
1071	372	32	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	87.21	85.68	88.49	87.98	89.55	86.99	87.26	87.18	87.88	87.80	88.27	86.99	87.97	86.66	87.00	86.67	86.82	87.74	87.50	88.64	89.37	89.69	91.14	89.71	87.49
1120	372	32	0.1	64	0.00010	48	0.000000e+00	0.000000e+00	86.19	86.22	85.39	83.88	88.43	88.76	87.99	88.44	86.95	87.88	90.39	87.24	87.85	85.65	87.93	86.02	87.97	84.71	87.91	90.74	88.01	89.89	88.12	89.27	87.35
1123	372	32	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	88.99	85.29	87.42	87.70	85.80	90.65	86.24	88.33	86.26	86.88	87.69	88.38	88.70	87.21	88.69	86.83	87.12	88.26	88.04	88.31	89.53	90.34	90.14	89.97	89.32
1132	372	32	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	82.34	87.24	85.83	84.83	84.32	87.83	84.92	86.55	87.60	86.91	86.95	90.15	85.84	85.01	88.62	88.54	85.45	85.87	87.78	92.33	89.65	89.59	89.37	87.15	87.88
1135	372	32	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	86.14	87.21	88.77	85.25	88.44	87.57	84.43	86.61	85.73	87.44	87.96	87.76	87.43	87.85	89.03	87.65	86.69	86.97	90.06	91.21	88.72	90.81	88.03	91.39	90.79
1184	372	32	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	83.84	86.94	84.75	88.37	83.90	85.93	84.25	86.79	82.74	85.74	85.81	86.50	85.84	89.04	89.22	87.42	85.56	85.56	87.57	86.29	89.14	87.84	87.81	88.19	87.76
1187	372	32	0.1	16	0.00001	24	0.000000e+00	1.000000e-08	86.00	89.37	86.18	88.09	85.35	86.99	85.06	86.32	87.47	85.81	86.37	88.88	89.35	86.76	88.29	87.47	85.60	86.86	87.92	90.47	87.65	87.99	87.98	89.12	88.54
1196	372	32	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	85.15	83.40	83.44	83.05	87.05	87.19	83.84	85.40	85.51	86.10	88.04	87.68	89.75	87.02	88.74	86.11	87.85	87.24	88.71	90.36	87.80	89.29	88.25	89.28	90.64
1199	372	32	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	84.80	85.76	86.89	84.13	86.20	86.34	86.29	86.71	87.36	86.08	88.41	88.83	86.34	86.22	86.11	87.46	89.71	88.02	85.09	87.38	89.01	87.74	87.69	89.90	88.30
1248	372	32	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	85.11	85.26	85.55	83.69	86.62	87.52	86.49	86.53	85.96	85.14	88.01	88.12	88.55	86.44	86.22	87.02	84.47	86.57	90.54	88.03	90.00	88.30	87.97	86.99	88.08
1251	372	32	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	86.01	86.78	87.23	86.34	86.63	87.94	86.29	85.97	86.33	86.17	88.84	88.72	85.37	87.61	87.65	85.93	86.55	87.98	88.98	88.80	91.31	87.95	88.41	89.36	88.74
1260	372	32	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	87.12	86.38	87.67	86.26	85.27	86.80	85.87	86.35	86.74	87.21	88.09	89.14	86.12	83.98	89.72	86.13	85.83	87.81	88.80	87.80	88.11	88.26	89.18	88.52	88.53
1263	372	32	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	85.60	87.79	87.08	87.15	86.89	87.05	86.70	87.22	87.18	86.89	89.06	89.00	88.68	85.48	85.40	86.77	87.58	87.40	88.42	88.09	88.68	90.52	87.08	89.88	89.30
1312	372	32	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	83.95	84.79	85.96	85.60	87.36	85.49	87.68	88.85	85.46	85.94	86.29	87.66	86.65	86.65	87.61	88.11	87.56	86.09	89.18	90.21	88.59	74.49	85.34	89.24	89.14
1315	372	32	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	85.10	86.25	86.59	85.16	86.37	88.25	86.80	86.56	86.48	87.55	86.90	87.82	85.54	87.53	86.17	87.03	87.73	87.36	86.79	89.25	89.96	88.94	88.99	89.88	87.74
1324	372	32	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	85.43	88.18	88.08	86.56	84.62	87.21	86.19	86.31	87.31	88.32	88.56	87.75	87.61	89.22	87.36	89.85	85.79	87.26	89.76	89.20	88.95	87.80	89.53	87.47	88.24
1327	372	32	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	85.33	86.79	86.24	86.12	85.69	87.93	86.82	87.54	85.88	90.08	88.42	87.90	88.21	87.53	86.26	87.62	88.40	87.56	87.86	88.68	89.91	89.05	88.02	86.67	91.92
1376	372	32	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	85.67	85.55	88.37	85.01	85.72	88.14	84.23	87.33	87.23	88.23	87.23	88.04	86.58	86.07	87.16	86.80	87.63	85.43	85.47	86.70	87.77	86.88	87.90	89.19	86.81
1379	372	32	0.1	16	0.00001	36	0.000000e+00	1.000000e-08	84.02	86.02	87.68	84.08	85.62	85.31	85.37	86.03	83.72	85.43	88.22	90.89	86.33	88.47	84.94	88.80	85.11	87.52	87.80	89.15	87.03	87.15	85.12	88.63	88.14
1388	372	32	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	86.26	85.60	86.93	85.47	87.53	85.92	84.45	86.99	85.88	87.05	88.57	87.43	85.84	87.64	88.23	88.65	87.94	85.40	89.85	87.77	88.61	89.57	86.54	87.54	88.12
1391	372	32	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	85.31	88.10	85.53	85.69	84.48	85.40	84.22	87.88	84.17	87.13	87.54	87.11	88.34	87.67	87.13	87.69	85.81	86.39	90.01	88.13	87.71	89.36	88.86	88.34	87.51
1440	372	32	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	85.74	89.00	84.99	87.18	87.64	85.68	85.53	86.66	89.39	85.32	87.16	87.30	86.87	86.75	86.12	87.96	87.94	88.07	86.92	87.06	89.07	89.08	89.24	88.71	87.60
1443	372	32	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	87.69	87.20	87.25	86.39	85.08	84.80	86.32	85.42	87.29	85.47	87.25	86.65	87.17	85.68	87.45	86.57	87.04	86.76	88.67	86.79	87.24	87.02	88.44	88.21	88.64
1452	372	32	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	85.93	86.79	87.17	86.05	84.99	87.28	85.76	86.04	87.79	88.24	89.22	87.85	87.40	87.91	85.73	89.09	88.92	87.09	87.44	89.69	91.22	88.93	88.92	88.60	89.20
1455	372	32	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	86.28	85.07	83.86	86.64	86.86	86.90	84.81	85.53	86.06	85.27	87.40	86.40	84.50	86.17	88.10	85.53	86.45	87.43	87.58	87.26	89.16	88.41	88.83	89.45	87.35
1504	372	32	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	84.98	84.18	90.51	86.11	83.88	87.26	86.81	89.47	87.15	84.02	87.26	88.78	86.17	86.40	86.28	84.85	88.34	86.16	87.97	86.65	87.32	89.41	89.05	82.34	88.30
1507	372	32	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	86.18	85.81	85.81	87.76	86.67	87.76	87.18	86.30	86.95	86.07	86.48	87.71	85.90	84.95	85.01	88.35	86.49	85.13	88.53	87.16	87.30	87.66	88.88	88.21	88.21
1516	372	32	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	84.56	86.61	86.65	85.44	84.89	88.21	85.05	89.36	85.87	85.14	86.68	87.08	87.68	84.73	85.34	87.30	86.52	86.05	87.77	88.62	88.75	88.95	86.24	88.71	88.61
1519	372	32	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	84.89	86.92	86.22	85.85	84.23	86.41	85.14	86.26	84.50	86.37	85.86	88.68	86.94	86.15	86.56	86.02	89.14	85.72	87.10	86.86	87.10	88.19	87.52	88.16	85.87
1568	372	32	0.1	16	0.00001	48	0.000000e+00	0.000000e+00	86.31	85.86	86.38	84.48	84.33	84.53	84.91	86.70	86.66	88.59	87.14	88.18	87.80	85.96	85.44	88.68	84.56	86.80	88.23	89.66	89.60	89.25	88.01	87.99	88.86
1571	372	32	0.1	16	0.00001	48	0.000000e+00	1.000000e-08	83.94	84.79	84.52	83.52	85.05	86.92	84.49	88.46	85.24	83.93	88.83	85.33	86.51	85.61	89.61	85.73	85.59	84.23	90.25	86.83	87.67	89.47	87.25	87.59	88.16

Test Data mAP Results Preview:

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_abs_values	mAP_test_abs_values_2	mAP_test_abs_values_3	mAP_test_abs_values_4	mAP_test_abs_values_5	mAP_test_abs_values_6	mAP_test_abs_values_7	mAP_test_abs_values_8	mAP_test_abs_values_9	mAP_test_abs_values_10	mAP_test_abs_values_11	mAP_test_abs_values_12	mAP_test_abs_values_13	mAP_test_abs_values_14	mAP_test_abs_values_15	mAP_test_abs_values_16	mAP_test_abs_values_17	mAP_test_abs_values_18	mAP_test_abs_values_19	mAP_test_abs_values_20	mAP_test_abs_values_21	mAP_test_abs_values_22	mAP_test_abs_values_23	mAP_test_abs_values_25	mAP_test_abs_values_26
32	372	32	0.1	16	0.00100	24	0.000000e+00	0.000000e+00	92.54	92.20	92.36	92.59	91.25	90.60	90.41	94.71	92.63	89.37	91.50	91.89	94.07	92.77	91.77	93.71	91.17	94.49	89.45	90.70	91.64	89.94	91.10	92.55	88.56
35	372	32	0.1	16	0.00100	24	0.000000e+00	1.000000e-08	91.48	93.62	93.40	93.06	91.67	92.46	93.40	92.21	92.64	91.99	81.07	91.06	91.88	93.07	94.59	94.65	94.82	48.12	90.92	91.77	92.03	93.83	91.55	90.67	92.81
44	372	32	0.1	16	0.00100	24	1.000000e-08	0.000000e+00	94.07	92.34	94.47	94.25	94.30	93.83	94.04	94.34	90.80	93.44	91.47	91.52	92.20	93.70	92.41	92.62	91.95	93.50	91.34	93.69	92.13	93.64	92.59	91.44	91.95
47	372	32	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	94.35	94.65	94.17	91.67	93.91	78.74	93.63	92.97	92.93	92.58	92.10	91.74	93.20	92.28	92.43	94.31	93.65	92.01	81.53	92.00	92.53	94.37	93.19	93.77	93.32
96	372	32	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	91.77	94.88	95.31	94.48	92.46	90.85	48.12	92.95	95.06	92.90	91.82	92.63	93.34	94.32	93.66	48.12	94.78	94.09	89.87	91.08	90.58	94.28	91.36	91.41	91.68
99	372	32	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	96.24	94.29	93.74	93.75	93.28	91.69	94.19	93.51	93.14	94.43	90.43	92.64	94.54	93.88	94.61	95.29	94.85	95.79	93.01	91.10	91.86	92.90	94.19	92.87	93.96
108	372	32	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	91.88	93.29	95.20	94.82	94.77	93.29	94.82	93.60	93.93	93.24	91.95	91.98	93.20	96.01	94.10	92.68	96.69	93.88	90.18	91.94	91.58	90.64	93.72	92.75	91.37
111	372	32	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	92.86	94.73	95.05	93.90	92.90	92.66	93.75	96.18	94.07	94.63	92.13	94.32	95.59	94.08	94.54	94.91	91.38	93.53	91.58	91.88	92.28	91.94	92.57	89.92	92.10
160	372	32	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	94.59	91.73	94.25	94.77	93.60	93.11	94.56	94.06	94.28	95.18	92.19	93.14	93.09	92.83	95.03	92.02	95.37	92.43	89.67	91.23	91.11	91.70	90.17	91.37	91.36
163	372	32	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	94.24	95.34	94.50	92.02	89.24	93.38	92.39	95.00	93.62	95.05	93.11	92.10	94.56	96.21	94.14	92.95	93.99	93.20	92.53	91.63	84.37	91.58	92.85	94.49	92.25
172	372	32	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	93.97	92.99	94.55	96.27	94.55	93.58	93.28	48.12	95.55	96.52	94.23	93.17	94.43	48.12	95.09	94.45	94.38	92.16	90.20	94.75	92.25	94.80	48.12	93.39	91.56
175	372	32	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	94.40	92.80	93.60	94.31	94.80	92.62	94.43	94.07	94.48	93.08	93.59	90.64	95.40	94.10	92.85	93.96	96.00	94.80	92.48	91.67	91.46	92.29	93.75	92.20	93.35
224	372	32	0.1	16	0.00100	36	0.000000e+00	0.000000e+00	91.29	92.46	91.94	92.73	94.60	91.25	94.04	92.72	92.95	93.03	90.93	91.30	93.22	93.49	91.97	93.14	90.19	91.49	88.03	88.14	93.56	93.57	90.53	91.99	90.77
227	372	32	0.1	16	0.00100	36	0.000000e+00	1.000000e-08	93.57	93.23	93.86	93.85	94.99	91.54	92.14	95.70	94.91	94.39	94.80	93.71	93.60	95.60	92.25	91.98	94.35	93.84	93.56	91.53	94.79	95.47	94.53	93.36	93.79
236	372	32	0.1	16	0.00100	36	1.000000e-08	0.000000e+00	92.99	94.94	91.49	95.47	94.25	89.87	89.83	92.69	93.51	90.38	92.26	90.09	92.33	93.24	93.16	93.83	91.99	93.74	91.95	91.58	92.03	91.59	92.45	91.98	92.03
239	372	32	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	92.84	92.83	94.52	93.68	91.83	91.45	94.90	91.88	93.99	94.34	93.25	92.31	93.09	94.52	94.03	93.41	93.33	92.26	92.25	92.38	91.83	93.80	92.63	93.06	91.69
288	372	32	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	95.06	93.89	92.64	94.34	94.13	91.76	93.33	94.22	92.87	94.49	92.52	91.94	94.20	95.06	93.75	94.27	48.12	92.87	48.12	92.64	93.43	93.01	93.44	93.87	91.50
291	372	32	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	92.43	95.68	93.62	95.51	94.80	92.05	94.04	94.44	94.08	93.79	92.37	93.32	94.12	94.80	94.96	93.30	94.76	95.08	91.29	94.78	94.35	94.24	92.77	90.59	93.95
300	372	32	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	95.01	94.75	95.33	93.50	95.13	91.94	95.01	92.03	93.59	93.02	93.30	93.46	95.49	94.52	95.68	93.92	95.13	93.59	91.81	93.36	94.08	90.11	48.12	92.75	93.46
303	372	32	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	95.11	93.42	94.63	94.56	93.30	92.48	94.02	95.14	93.46	94.51	92.55	91.63	94.48	93.47	95.04	95.06	95.21	95.35	94.11	92.94	93.80	91.28	92.75	93.91	90.00
352	372	32	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	93.25	94.03	93.55	93.53	95.14	93.15	94.27	92.71	93.14	94.83	93.87	93.80	94.01	95.06	94.39	94.84	93.37	94.20	93.10	92.51	91.74	92.38	92.81	92.47	93.69
355	372	32	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	93.75	94.79	94.20	94.84	95.83	92.59	93.83	93.61	94.21	95.29	93.85	93.60	93.84	93.40	93.16	95.02	94.36	95.09	94.61	92.35	91.36	93.56	91.42	92.15	93.06
364	372	32	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	94.50	93.06	93.93	92.03	94.04	91.86	94.61	92.76	94.36	94.54	93.39	93.60	94.14	93.15	91.54	96.43	91.38	94.74	93.42	91.49	92.82	91.70	91.59	92.30	89.95
367	372	32	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	94.87	92.21	94.68	94.05	94.75	91.29	94.84	93.67	94.24	94.48	91.32	93.56	93.74	94.35	93.96	95.81	94.51	94.17	94.28	93.32	92.49	93.75	94.89	93.98	92.30
416	372	32	0.1	16	0.00100	48	0.000000e+00	0.000000e+00	94.62	90.09	93.78	87.95	93.20	90.25	92.66	90.21	94.17	93.79	89.15	94.45	94.79	94.12	89.62	95.09	91.59	93.00	94.26	92.48	92.71	92.31	89.90	90.99	92.64
419	372	32	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	95.52	89.02	92.69	96.47	94.24	87.37	94.30	91.18	94.76	94.01	94.28	91.70	93.31	94.03	93.64	95.14	95.25	93.94	93.54	91.02	92.39	92.35	62.25	94.30	95.01
428	372	32	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	92.26	94.48	94.99	93.87	92.66	93.03	93.09	89.85	94.10	92.73	89.94	90.74	93.65	93.16	92.27	94.24	91.41	93.11	92.93	92.30	95.54	94.09	90.42	92.40	93.98
431	372	32	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	92.90	93.86	94.48	59.15	88.49	93.59	94.04	92.28	95.47	93.85	95.55	92.81	91.96	90.15	95.32	93.98	92.67	94.88	93.97	92.24	91.80	94.38	48.12	93.50	92.70
480	372	32	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	92.64	92.77	95.73	93.19	92.81	92.45	94.94	93.29	95.47	94.32	93.01	93.74	95.71	92.00	95.34	95.65	92.91	94.35	90.31	87.99	91.55	59.11	92.48	85.88	90.90
483	372	32	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	94.09	95.24	94.05	92.62	94.14	92.15	92.12	93.46	48.12	93.82	93.25	90.46	93.82	95.88	93.75	93.98	96.53	94.37	90.92	92.84	57.77	94.20	93.51	92.66	93.37
492	372	32	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	93.94	95.61	95.27	95.52	95.56	93.01	93.61	93.35	93.12	94.22	93.19	93.69	93.12	93.70	95.25	94.24	93.76	90.84	94.29	93.61	93.22	91.78	87.80	93.49	93.08
495	372	32	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	93.10	96.04	92.29	92.51	88.44	94.01	94.30	94.26	93.53	95.03	94.08	93.63	91.25	94.55	92.24	95.12	94.07	93.49	92.78	92.48	92.64	93.47	91.67	93.16	93.96
544	372	32	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	93.22	93.93	95.50	91.66	92.10	93.75	92.59	94.76	94.18	93.18	92.84	94.00	90.63	92.97	92.10	91.50	91.95	93.05	91.22	93.60	91.73	93.61	94.83	93.71	94.87
547	372	32	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	92.16	92.18	93.59	93.63	94.31	91.73	95.74	92.59	92.72	94.42	91.45	93.78	95.29	93.21	92.47	93.78	93.16	94.60	92.95	92.26	92.80	93.70	93.84	92.30	91.30
556	372	32	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	94.48	93.32	91.89	92.43	92.27	92.55	94.13	93.21	94.53	93.09	93.99	92.52	94.53	94.82	92.99	94.64	93.65	95.26	91.49	92.66	91.99	93.32	94.62	94.43	92.69
559	372	32	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	94.03	94.21	92.87	95.00	91.92	90.50	94.40	93.53	94.88	93.08	92.90	94.50	96.31	91.87	92.16	92.79	93.68	94.78	94.02	93.26	93.73	93.68	92.93	91.95	91.38
608	372	32	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	92.55	90.78	91.84	94.34	92.79	91.71	48.12	94.31	90.56	94.18	91.23	91.48	89.85	94.21	92.20	90.83	91.21	92.87	90.40	90.63	92.01	88.41	91.38	90.38	91.91
611	372	32	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	90.60	93.50	94.14	95.84	93.36	92.39	94.74	94.19	95.18	93.96	93.31	89.50	90.58	90.31	92.60	94.41	94.77	92.73	92.72	91.33	94.64	91.13	91.24	93.39	91.75
620	372	32	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	96.26	94.04	94.91	95.30	95.27	91.79	93.64	91.93	95.29	93.93	92.41	93.74	94.28	96.23	95.19	87.22	91.45	94.88	93.47	93.29	93.18	90.06	93.03	92.83	92.52
623	372	32	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	94.48	93.21	94.10	92.85	92.10	94.58	93.26	95.42	91.01	95.70	94.04	92.54	92.92	94.92	93.84	92.91	94.44	92.26	90.67	88.42	93.11	94.42	93.17	91.62	89.06
672	372	32	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	94.65	91.19	93.54	89.44	92.91	91.59	94.35	91.13	93.30	94.52	93.46	92.62	93.22	94.31	93.57	93.46	94.13	93.97	92.91	92.99	91.75	91.85	93.14	91.12	91.99
675	372	32	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	94.15	93.78	94.18	94.43	95.08	91.62	95.05	93.82	94.67	92.47	92.96	93.17	93.69	95.72	92.17	94.08	95.08	92.81	92.89	93.36	90.40	91.44	92.90	91.82	90.63
684	372	32	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	92.38	92.93	94.06	93.43	94.26	92.75	94.45	92.50	94.17	95.70	94.87	93.88	93.06	95.45	94.20	93.44	91.71	91.63	92.77	92.51	92.35	91.54	92.82	91.30	89.67
687	372	32	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	91.48	93.32	93.14	93.99	94.56	91.07	94.98	93.16	90.70	91.72	94.03	93.43	93.25	94.91	91.98	93.62	93.00	92.36	91.77	92.57	92.18	92.17	92.03	93.04	91.90
736	372	32	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	93.81	93.93	92.36	91.49	93.18	93.11	92.28	94.79	93.06	94.36	92.48	93.12	94.26	94.33	91.95	94.44	93.11	93.33	90.81	91.42	90.94	91.94	92.55	93.03	92.60
739	372	32	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	94.13	93.08	95.22	93.19	48.12	93.79	94.52	92.88	92.19	92.60	91.04	92.06	89.44	94.77	94.66	93.70	94.76	94.09	91.44	90.47	94.16	91.96	91.75	90.94	89.82
748	372	32	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	92.74	94.24	93.46	94.97	92.51	91.79	93.58	94.91	92.96	96.04	93.92	92.15	94.29	94.91	95.29	93.27	94.19	94.54	93.80	94.11	91.32	93.27	93.80	92.00	92.40
751	372	32	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	95.93	92.72	93.26	94.54	93.64	93.43	93.50	91.85	93.28	93.81	94.28	91.40	93.42	91.02	93.73	95.21	94.77	91.76	93.54	91.59	91.92	90.60	91.68	91.94	92.34
800	372	32	0.1	16	0.00010	36	0.000000e+00	0.000000e+00	92.39	93.31	91.65	91.54	93.18	89.86	93.45	94.48	93.30	95.91	92.11	90.91	90.31	93.17	92.77	91.75	93.13	90.40	90.75	89.63	89.74	93.37	92.90	94.60	94.48
803	372	32	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	94.68	95.50	93.51	95.81	93.32	91.59	93.47	94.89	93.39	92.39	92.25	93.23	92.10	93.39	91.79	93.30	94.29	93.07	90.88	86.93	90.29	91.69	92.23	93.97	94.58
812	372	32	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	93.01	92.86	94.36	93.34	92.54	92.52	93.41	95.59	93.42	95.28	91.29	92.18	90.69	93.81	94.17	93.96	96.74	93.86	92.30	93.89	93.90	93.60	93.93	93.94	94.07
815	372	32	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	90.65	91.61	93.13	93.01	92.48	91.07	92.72	93.61	91.27	91.98	91.08	91.01	92.65	92.94	92.09	92.74	93.29	90.90	92.08	87.13	91.58	92.07	91.82	93.51	89.12
864	372	32	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	93.83	94.00	93.52	94.27	93.47	92.78	93.31	91.27	48.12	93.81	92.47	93.10	93.16	94.39	92.19	94.66	94.22	94.56	91.50	89.61	90.89	91.06	92.32	91.87	91.89
867	372	32	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	92.62	94.45	93.11	94.05	92.65	91.89	95.53	95.57	93.90	94.03	91.14	92.91	93.62	93.97	92.29	92.93	93.81	94.03	92.18	93.70	93.41	90.63	92.59	91.97	91.58
876	372	32	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	94.33	91.61	94.47	94.61	95.07	93.82	93.89	95.38	92.96	94.66	94.23	92.95	93.50	93.73	91.78	93.71	95.56	95.54	91.51	93.16	90.30	93.02	91.25	92.72	93.94
879	372	32	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	94.52	94.16	95.87	93.67	95.20	93.35	92.62	93.14	93.82	94.25	91.35	93.65	94.87	93.91	95.32	95.54	95.41	95.33	91.47	91.20	92.30	95.53	92.21	93.39	92.27
928	372	32	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	93.59	92.88	94.32	93.58	93.79	92.01	96.49	93.74	95.48	95.24	93.50	93.78	93.88	94.76	93.41	94.21	96.12	95.81	91.59	92.72	92.88	92.51	92.59	91.97	94.88
931	372	32	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	94.17	95.07	95.47	93.96	95.79	93.92	93.97	94.40	93.82	93.43	93.01	92.42	93.93	93.41	91.83	94.42	94.84	93.22	92.88	90.32	92.88	92.89	92.59	91.73	91.81
940	372	32	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	93.88	94.47	93.77	95.91	95.00	93.76	93.83	93.75	94.29	94.38	93.03	95.12	95.53	93.56	93.69	93.34	93.69	95.38	92.95	93.97	91.91	91.98	93.97	94.39	90.02
943	372	32	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	94.05	94.51	94.53	95.99	95.49	92.20	92.03	92.15	94.67	94.63	93.71	94.31	93.70	94.12	92.65	93.92	94.95	94.74	94.10	93.64	93.60	91.63	92.36	92.29	93.53
992	372	32	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	95.52	91.38	59.72	84.98	65.81	92.46	92.85	93.22	93.10	94.50	95.04	93.18	91.69	94.13	90.44	94.19	89.70	93.88	92.68	92.59	94.04	93.57	91.10	93.32	78.89
995	372	32	0.1	16	0.00010	48	0.000000e+00	1.000000e-08	91.74	89.24	89.69	95.94	93.00	91.22	94.58	94.52	92.70	92.54	92.13	92.46	95.29	91.84	93.66	93.70	92.51	95.99	92.14	91.90	91.82	93.51	91.86	91.31	94.32
1004	372	32	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	91.91	95.53	84.17	94.45	92.94	94.35	94.69	94.04	95.07	94.52	91.89	92.19	85.35	95.42	80.29	94.96	95.01	95.10	92.43	93.70	94.58	94.80	92.56	92.87	87.86
1007	372	32	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	93.93	94.41	94.33	95.44	91.93	93.22	93.99	93.48	92.77	93.71	91.67	94.62	94.25	92.74	92.15	91.52	93.72	92.32	93.40	93.09	93.67	93.05	89.94	93.65	92.47
1056	372	32	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	94.47	58.74	94.02	95.03	91.78	90.95	95.34	92.87	94.48	93.71	92.47	92.00	92.77	93.54	93.13	93.04	94.18	91.90	93.48	93.26	61.54	92.88	94.90	90.74	91.13
1059	372	32	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	93.50	94.49	95.84	94.58	94.74	91.88	93.05	95.50	94.72	93.83	92.63	93.41	95.14	90.84	92.02	94.80	92.96	94.11	93.33	90.77	93.56	91.70	92.99	85.21	91.60
1068	372	32	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	94.53	92.20	95.85	92.63	95.20	92.51	95.14	93.77	94.01	93.84	91.32	93.29	95.75	92.11	90.58	95.35	95.40	94.08	94.06	93.13	92.86	93.28	94.04	93.57	93.03
1071	372	32	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	94.26	93.58	95.26	95.27	94.41	92.67	93.13	94.79	93.76	94.20	92.42	93.58	94.05	93.31	93.45	94.08	93.73	94.78	93.32	92.63	94.76	93.40	59.16	91.82	92.89
1120	372	32	0.1	64	0.00010	48	0.000000e+00	0.000000e+00	93.37	93.32	94.47	94.70	92.84	92.41	94.56	93.56	94.72	95.50	90.69	93.20	95.03	95.67	93.29	92.99	92.09	90.63	93.76	90.31	87.51	94.38	94.05	92.37	91.06
1123	372	32	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	93.62	95.56	94.62	95.98	93.28	93.15	94.36	94.21	90.55	94.30	92.83	92.95	94.24	94.32	92.92	92.19	93.76	93.30	92.21	91.82	92.93	94.06	93.71	91.27	93.49
1132	372	32	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	93.87	93.30	93.05	92.37	93.70	91.83	93.25	93.79	94.57	93.96	93.85	92.04	93.97	92.73	92.58	94.04	93.83	92.57	91.50	92.61	95.14	94.08	94.99	94.62	87.60
1135	372	32	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	92.27	95.17	93.79	93.82	94.48	91.68	94.70	92.62	95.45	92.59	93.04	94.49	93.15	92.86	91.70	95.94	93.29	93.75	92.62	93.04	93.96	94.52	93.33	91.52	90.15
1184	372	32	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	95.69	94.11	90.79	91.02	92.72	91.26	90.71	92.36	92.53	91.32	91.64	92.02	92.74	93.87	92.20	88.79	94.06	90.80	91.68	90.69	93.56	91.23	91.62	90.13	89.42
1187	372	32	0.1	16	0.00001	24	0.000000e+00	1.000000e-08	93.41	93.43	93.57	93.92	95.97	93.58	92.79	93.31	94.14	91.95	90.28	90.54	95.54	91.95	91.88	92.30	94.92	92.82	92.31	90.95	92.02	94.11	92.39	92.66	89.58
1196	372	32	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	92.85	93.40	94.42	93.40	92.91	91.61	94.32	94.89	93.35	93.96	94.17	90.70	93.63	95.07	93.83	91.71	93.24	92.28	93.50	92.07	92.40	95.36	94.13	92.81	93.80
1199	372	32	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	91.89	95.32	91.37	94.82	92.97	91.35	94.75	95.55	95.17	93.12	91.13	92.81	90.99	93.20	94.00	95.24	84.38	94.12	90.28	93.78	93.09	93.66	93.57	90.17	93.13
1248	372	32	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	93.12	93.59	93.72	93.64	93.23	90.07	92.64	93.61	94.11	94.83	92.42	93.71	94.27	95.05	91.41	93.73	93.76	92.98	90.42	91.19	93.26	91.38	92.52	92.23	91.84
1251	372	32	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	95.11	92.53	95.55	95.16	93.85	94.03	94.41	92.07	94.51	93.42	92.55	91.59	93.51	95.19	94.72	91.92	94.28	94.18	91.56	91.46	92.31	91.71	92.61	92.48	91.59
1260	372	32	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	92.88	94.45	95.52	90.67	93.58	92.50	93.09	93.57	93.46	93.08	94.77	93.14	90.79	95.07	94.83	94.46	94.72	95.26	89.65	93.55	92.25	92.07	92.46	90.73	93.48
1263	372	32	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	92.24	91.34	93.26	92.89	93.62	90.76	93.34	91.42	94.12	94.13	93.20	91.00	92.69	94.40	93.12	89.32	95.07	93.08	92.03	91.87	92.65	93.02	93.84	91.01	91.14
1312	372	32	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	92.67	93.78	93.03	93.57	91.01	92.14	93.04	91.88	93.90	93.20	92.43	92.92	93.42	95.67	91.26	94.35	94.35	92.33	92.64	91.42	92.33	88.10	92.59	92.37	92.70
1315	372	32	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	92.98	94.03	95.40	92.81	95.78	92.23	92.53	95.18	93.85	94.94	92.85	93.72	90.94	94.29	93.74	95.58	95.83	95.02	94.01	92.86	92.91	91.28	92.04	91.79	93.68
1324	372	32	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	93.77	92.24	94.35	94.65	91.67	93.58	92.19	95.59	94.99	94.04	91.54	93.31	96.14	94.03	92.89	94.22	94.27	92.88	92.70	91.54	92.77	94.03	92.88	91.70	92.11
1327	372	32	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	95.61	93.88	93.79	91.83	95.09	93.14	94.62	95.47	91.84	95.14	93.15	93.30	93.50	92.45	93.60	92.66	95.57	93.91	91.36	92.38	92.89	93.63	93.23	91.80	89.53
1376	372	32	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	92.94	94.31	91.99	95.13	93.67	92.17	92.94	93.11	93.43	96.13	92.16	91.78	95.09	92.74	93.20	90.90	91.23	93.75	89.00	89.91	93.70	92.05	94.92	94.21	92.19
1379	372	32	0.1	16	0.00001	36	0.000000e+00	1.000000e-08	94.06	95.54	93.73	96.69	93.83	92.29	94.08	95.31	94.33	94.16	93.29	80.96	94.01	93.76	93.30	93.92	95.86	93.35	92.46	90.53	94.48	93.38	90.77	92.85	91.92
1388	372	32	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	92.32	93.26	93.43	95.35	93.09	93.35	92.41	91.72	94.86	93.67	93.16	93.16	94.02	94.38	95.41	93.69	94.55	90.73	92.87	92.55	94.68	89.85	92.98	91.12	93.54
1391	372	32	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	93.84	92.61	92.57	95.36	93.24	94.27	92.41	93.53	94.37	95.15	91.97	92.96	93.49	92.24	95.03	94.06	94.99	93.61	93.26	91.46	94.60	93.17	92.45	92.70	93.09
1440	372	32	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	93.17	94.12	92.75	92.46	85.23	92.12	94.78	92.24	95.16	94.49	92.14	91.31	93.46	95.29	93.75	94.14	94.59	88.98	92.87	92.60	92.03	92.96	93.55	91.44	92.60
1443	372	32	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	93.92	93.06	92.99	93.89	94.29	92.26	93.41	94.51	94.81	93.54	93.65	93.22	94.35	93.93	91.75	93.89	95.55	94.22	93.40	92.07	92.81	90.48	91.76	90.04	91.65
1452	372	32	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	93.13	95.05	94.69	95.44	94.98	92.47	95.77	95.05	95.43	95.01	91.35	93.86	93.54	94.40	93.19	94.88	95.26	95.35	93.28	91.71	91.90	91.84	92.83	93.20	91.62
1455	372	32	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	95.62	95.98	93.77	95.58	94.82	92.98	95.80	95.09	95.10	94.76	95.11	93.80	94.00	94.41	92.36	92.90	95.10	92.48	94.75	91.18	93.76	93.13	92.19	94.19	93.49
1504	372	32	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	93.85	95.55	95.06	94.11	92.14	93.82	93.79	93.84	95.59	94.67	92.95	95.28	95.60	94.15	95.65	95.13	94.97	96.05	92.61	93.14	93.44	93.37	92.30	94.64	92.54
1507	372	32	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	92.69	95.57	96.19	95.69	94.48	92.96	95.41	93.44	94.35	93.70	94.05	93.79	93.76	94.19	93.91	94.49	96.80	93.53	93.70	92.81	92.91	93.72	93.22	91.07	92.12
1516	372	32	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	94.41	94.18	96.01	94.96	95.48	92.44	94.39	93.13	95.32	93.90	94.08	93.46	93.92	93.53	94.57	93.76	95.37	95.18	93.60	93.49	92.23	92.55	93.41	92.00	94.18
1519	372	32	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	92.28	92.65	94.77	92.46	96.35	92.19	94.43	95.35	91.66	92.89	92.41	93.51	93.00	94.59	92.75	94.67	95.54	93.37	92.60	92.67	93.02	93.49	92.21	89.64	92.79
1568	372	32	0.1	16	0.00001	48	0.000000e+00	0.000000e+00	95.03	71.60	93.21	93.94	89.49	92.93	92.57	94.18	90.59	93.84	92.11	92.35	93.87	86.64	95.24	90.48	93.51	91.49	91.23	94.43	92.59	92.06	93.26	92.63	62.35
1571	372	32	0.1	16	0.00001	48	0.000000e+00	1.000000e-08	93.64	94.94	93.39	94.20	94.78	93.74	94.39	94.34	95.48	92.47	93.03	93.30	95.81	93.72	93.78	92.66	94.61	92.51	91.66	91.10	93.47	92.28	93.48	90.21	91.64

Model performance on validation data (T-test)

Statistical Significance:

Using the global mean maximum row as the reference, we perform the right-tailed t-test to identify significant hyperparameters.

pvalues = []
pvalues_real = []

for _,index in enumerate(dt_valid_sub.index):
  _, p_value = stats.ttest_ind(max_row[:-1], dt_valid_sub.loc[index][:-1], alternative='greater')
  pvalues_real.append(p_value)
  pvalues.append((p_value >= 0.05)*1)

  

dt_valid_sub['pvalues'] = pvalues_real
dt_valid_sub['sig'] = pvalues


dt_valid_sub1 = dt_valid_sub.query('sig == 1')


max_indexx = dt_valid_sub1.iloc[:, :-3].mean().idxmax()

#print(sum(pvalues),max_indexx)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_abs_values	mAP_valid_abs_values_2	mAP_valid_abs_values_3	mAP_valid_abs_values_4	mAP_valid_abs_values_5	mAP_valid_abs_values_6	mAP_valid_abs_values_7	mAP_valid_abs_values_8	mAP_valid_abs_values_9	mAP_valid_abs_values_10	mAP_valid_abs_values_11	mAP_valid_abs_values_12	mAP_valid_abs_values_13	mAP_valid_abs_values_14	mAP_valid_abs_values_15	mAP_valid_abs_values_16	mAP_valid_abs_values_17	mAP_valid_abs_values_18	mAP_valid_abs_values_19	mAP_valid_abs_values_20	mAP_valid_abs_values_21	mAP_valid_abs_values_22	mAP_valid_abs_values_23	mAP_valid_abs_values_25	mAP_valid_abs_values_26
431	372	32	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	85.72	85.81	86.24	87.63	84.94	87.47	84.27	85.81	85.83	87.51	88.27	89.92	87.52	85.80	86.03	86.76	90.59	85.74	89.70	87.99	90.48	88.20	43.72	87.94	89.55
1452	372	32	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	85.93	86.79	87.17	86.05	84.99	87.28	85.76	86.04	87.79	88.24	89.22	87.85	87.40	87.91	85.73	89.09	88.92	87.09	87.44	89.69	91.22	88.93	88.92	88.60	89.20
687	372	32	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	86.32	86.76	87.05	87.52	86.74	87.02	86.38	87.31	88.17	86.68	88.64	89.54	87.36	87.10	87.99	88.63	86.43	87.45	89.60	88.46	89.09	89.80	90.05	88.30	88.81
1059	372	32	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	86.70	85.32	87.35	86.52	86.28	87.35	87.73	86.71	86.79	87.70	89.10	88.22	86.50	86.82	87.90	88.35	87.90	87.53	90.55	88.27	87.58	89.38	90.06	89.91	89.77
1068	372	32	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	87.27	87.65	87.08	85.49	87.92	86.72	86.20	87.75	88.60	89.02	88.44	88.49	89.48	85.41	87.67	88.37	87.00	87.65	89.04	89.17	88.42	89.17	88.13	90.03	90.25
1071	372	32	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	87.21	85.68	88.49	87.98	89.55	86.99	87.26	87.18	87.88	87.80	88.27	86.99	87.97	86.66	87.00	86.67	86.82	87.74	87.50	88.64	89.37	89.69	91.14	89.71	87.49
99	372	32	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	86.47	87.08	84.36	86.55	86.57	88.08	87.45	86.88	87.08	86.79	89.07	88.21	86.33	85.30	88.03	86.91	86.80	87.80	89.13	89.74	90.60	89.02	89.72	88.91	89.06
108	372	32	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	87.26	86.15	86.98	86.80	87.01	87.63	85.50	86.17	89.16	88.01	88.29	88.50	89.30	86.56	86.96	86.63	86.66	88.81	88.92	88.35	88.92	87.39	87.13	89.87	90.30
111	372	32	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	84.54	84.88	87.87	86.70	88.55	87.58	87.46	86.24	89.87	88.57	88.20	89.83	89.44	87.26	87.25	86.37	87.98	89.71	87.20	87.60	90.87	86.13	88.69	88.61	89.97
480	372	32	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	86.47	85.23	87.09	86.92	86.11	86.22	84.78	86.88	86.61	87.75	89.84	89.06	86.93	85.81	87.06	87.97	85.70	88.13	89.07	90.33	87.81	92.61	89.41	88.31	89.96
483	372	32	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	85.50	86.98	87.25	87.08	87.69	90.44	83.91	86.28	43.72	87.61	89.66	89.77	86.61	87.63	88.08	87.40	85.53	87.68	90.02	88.12	90.87	88.35	88.06	87.03	89.98
495	372	32	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	85.53	86.15	85.32	86.37	83.24	89.48	90.21	88.14	87.93	88.02	86.47	89.64	90.62	86.92	91.48	87.75	87.88	87.74	88.88	91.25	89.24	88.83	90.97	87.81	89.87
1324	372	32	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	85.43	88.18	88.08	86.56	84.62	87.21	86.19	86.31	87.31	88.32	88.56	87.75	87.61	89.22	87.36	89.85	85.79	87.26	89.76	89.20	88.95	87.80	89.53	87.47	88.24
1327	372	32	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	85.33	86.79	86.24	86.12	85.69	87.93	86.82	87.54	85.88	90.08	88.42	87.90	88.21	87.53	86.26	87.62	88.40	87.56	87.86	88.68	89.91	89.05	88.02	86.67	91.92
739	372	32	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	86.19	86.57	85.85	88.08	43.72	87.68	86.76	86.56	88.23	87.70	87.34	87.72	86.60	87.38	85.76	86.87	84.97	88.22	85.92	88.71	89.70	88.69	88.98	87.98	89.65
1123	372	32	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	88.99	85.29	87.42	87.70	85.80	90.65	86.24	88.33	86.26	86.88	87.69	88.38	88.70	87.21	88.69	86.83	87.12	88.26	88.04	88.31	89.53	90.34	90.14	89.97	89.32
1135	372	32	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	86.14	87.21	88.77	85.25	88.44	87.57	84.43	86.61	85.73	87.44	87.96	87.76	87.43	87.85	89.03	87.65	86.69	86.97	90.06	91.21	88.72	90.81	88.03	91.39	90.79
547	372	32	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	86.14	87.67	86.50	87.07	87.51	89.98	88.49	87.38	87.17	89.53	88.39	89.59	88.90	88.02	86.27	90.24	85.71	85.20	92.18	90.23	88.57	88.39	89.98	91.55	91.02
556	372	32	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	86.12	85.43	85.71	84.21	87.99	87.76	87.30	86.55	85.79	91.00	88.08	91.82	87.31	87.38	87.00	87.10	85.69	85.60	91.76	87.93	87.94	89.74	87.72	88.83	87.69

Size of the All data:  (98, 28)
Size of the Sig data:  (19, 28)

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params1 = dt_valid.loc[dt_valid_sub1.index][['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
optimal_params = optimal_params1.reset_index()
optimal_params.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_1 = pd.merge(optimal_params, dt_test, on=['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg'])
test_data_1_sub = test_data_1[list(test_data_1.columns[test_data_1.columns.str.startswith('mAP_test')])]

Equivalent test data

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_abs_values	mAP_test_abs_values_2	mAP_test_abs_values_3	mAP_test_abs_values_4	mAP_test_abs_values_5	mAP_test_abs_values_6	mAP_test_abs_values_7	mAP_test_abs_values_8	mAP_test_abs_values_9	mAP_test_abs_values_10	mAP_test_abs_values_11	mAP_test_abs_values_12	mAP_test_abs_values_13	mAP_test_abs_values_14	mAP_test_abs_values_15	mAP_test_abs_values_16	mAP_test_abs_values_17	mAP_test_abs_values_18	mAP_test_abs_values_19	mAP_test_abs_values_20	mAP_test_abs_values_21	mAP_test_abs_values_22	mAP_test_abs_values_23	mAP_test_abs_values_25	mAP_test_abs_values_26
18	372	32	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	92.90	93.86	94.48	59.15	88.49	93.59	94.04	92.28	95.47	93.85	95.55	92.81	91.96	90.15	95.32	93.98	92.67	94.88	93.97	92.24	91.80	94.38	48.12	93.50	92.70
10	372	32	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	93.13	95.05	94.69	95.44	94.98	92.47	95.77	95.05	95.43	95.01	91.35	93.86	93.54	94.40	93.19	94.88	95.26	95.35	93.28	91.71	91.90	91.84	92.83	93.20	91.62
7	372	32	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	91.48	93.32	93.14	93.99	94.56	91.07	94.98	93.16	90.70	91.72	94.03	93.43	93.25	94.91	91.98	93.62	93.00	92.36	91.77	92.57	92.18	92.17	92.03	93.04	91.90
8	372	32	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	93.50	94.49	95.84	94.58	94.74	91.88	93.05	95.50	94.72	93.83	92.63	93.41	95.14	90.84	92.02	94.80	92.96	94.11	93.33	90.77	93.56	91.70	92.99	85.21	91.60
3	372	32	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	94.53	92.20	95.85	92.63	95.20	92.51	95.14	93.77	94.01	93.84	91.32	93.29	95.75	92.11	90.58	95.35	95.40	94.08	94.06	93.13	92.86	93.28	94.04	93.57	93.03
5	372	32	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	94.26	93.58	95.26	95.27	94.41	92.67	93.13	94.79	93.76	94.20	92.42	93.58	94.05	93.31	93.45	94.08	93.73	94.78	93.32	92.63	94.76	93.40	59.16	91.82	92.89
14	372	32	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	96.24	94.29	93.74	93.75	93.28	91.69	94.19	93.51	93.14	94.43	90.43	92.64	94.54	93.88	94.61	95.29	94.85	95.79	93.01	91.10	91.86	92.90	94.19	92.87	93.96
9	372	32	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	91.88	93.29	95.20	94.82	94.77	93.29	94.82	93.60	93.93	93.24	91.95	91.98	93.20	96.01	94.10	92.68	96.69	93.88	90.18	91.94	91.58	90.64	93.72	92.75	91.37
6	372	32	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	92.86	94.73	95.05	93.90	92.90	92.66	93.75	96.18	94.07	94.63	92.13	94.32	95.59	94.08	94.54	94.91	91.38	93.53	91.58	91.88	92.28	91.94	92.57	89.92	92.10
13	372	32	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	92.64	92.77	95.73	93.19	92.81	92.45	94.94	93.29	95.47	94.32	93.01	93.74	95.71	92.00	95.34	95.65	92.91	94.35	90.31	87.99	91.55	59.11	92.48	85.88	90.90
16	372	32	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	94.09	95.24	94.05	92.62	94.14	92.15	92.12	93.46	48.12	93.82	93.25	90.46	93.82	95.88	93.75	93.98	96.53	94.37	90.92	92.84	57.77	94.20	93.51	92.66	93.37
1	372	32	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	93.10	96.04	92.29	92.51	88.44	94.01	94.30	94.26	93.53	95.03	94.08	93.63	91.25	94.55	92.24	95.12	94.07	93.49	92.78	92.48	92.64	93.47	91.67	93.16	93.96
11	372	32	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	93.77	92.24	94.35	94.65	91.67	93.58	92.19	95.59	94.99	94.04	91.54	93.31	96.14	94.03	92.89	94.22	94.27	92.88	92.70	91.54	92.77	94.03	92.88	91.70	92.11
12	372	32	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	95.61	93.88	93.79	91.83	95.09	93.14	94.62	95.47	91.84	95.14	93.15	93.30	93.50	92.45	93.60	92.66	95.57	93.91	91.36	92.38	92.89	93.63	93.23	91.80	89.53
17	372	32	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	94.13	93.08	95.22	93.19	48.12	93.79	94.52	92.88	92.19	92.60	91.04	92.06	89.44	94.77	94.66	93.70	94.76	94.09	91.44	90.47	94.16	91.96	91.75	90.94	89.82
2	372	32	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	93.62	95.56	94.62	95.98	93.28	93.15	94.36	94.21	90.55	94.30	92.83	92.95	94.24	94.32	92.92	92.19	93.76	93.30	92.21	91.82	92.93	94.06	93.71	91.27	93.49
4	372	32	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	92.27	95.17	93.79	93.82	94.48	91.68	94.70	92.62	95.45	92.59	93.04	94.49	93.15	92.86	91.70	95.94	93.29	93.75	92.62	93.04	93.96	94.52	93.33	91.52	90.15
0	372	32	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	92.16	92.18	93.59	93.63	94.31	91.73	95.74	92.59	92.72	94.42	91.45	93.78	95.29	93.21	92.47	93.78	93.16	94.60	92.95	92.26	92.80	93.70	93.84	92.30	91.30
15	372	32	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	94.48	93.32	91.89	92.43	92.27	92.55	94.13	93.21	94.53	93.09	93.99	92.52	94.53	94.82	92.99	94.64	93.65	95.26	91.49	92.66	91.99	93.32	94.62	94.43	92.69

Size of the test data:  (19, 33)

Difference

diff = dt[['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
for i, cols in enumerate(zip(dt.columns,test_data_1.columns)):   
   diff[f'col_diff_{i}'] = (np.array(dt[cols[0]]) - np.array(test_data_1[cols[1]]))*-1

# Function to count negatives in a row
def count_negatives(row):
    return sum(1 for value in row if isinstance(value, (int, float)) and value < 0)

diff['negative_count'] = diff.apply(count_negatives, axis=1)
diff.sort_values(by =["negative_count"],inplace = True, ascending=False)

html_table = diff.to_html(index=True)

# Wrap in a scrollable div
scrollable_table = f"""
<div style="height: 400px; width: 100%; overflow-x: auto; overflow-y: auto;">
    {html_table}
</div>
"""
# Display the scrollable table
display(HTML(scrollable_table))

# 
# show(diff)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	col_diff_0	col_diff_1	col_diff_2	col_diff_3	col_diff_4	col_diff_5	col_diff_6	col_diff_7	col_diff_8	col_diff_9	col_diff_10	col_diff_11	col_diff_12	col_diff_13	col_diff_14	col_diff_15	col_diff_16	col_diff_17	col_diff_18	col_diff_19	col_diff_20	col_diff_21	col_diff_22	col_diff_23	col_diff_24	col_diff_25	col_diff_26	col_diff_27	col_diff_28	col_diff_29	col_diff_30	col_diff_31	col_diff_32	negative_count
480	372	32	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.17	7.54	8.64	6.27	6.70	6.23	10.16	6.41	8.86	6.57	3.17	4.68	8.78	6.19	8.28	7.68	7.21	6.22	1.24	-2.34	3.74	-33.50	3.07	-2.43	0.94	3
1071	372	32	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.05	7.90	6.77	7.29	4.86	5.68	5.87	7.61	5.88	6.40	4.15	6.59	6.08	6.65	6.45	7.41	6.91	7.04	5.82	3.99	5.39	3.71	-31.98	2.11	5.40	1
1135	372	32	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.13	7.96	5.02	8.57	6.04	4.11	10.27	6.01	9.72	5.15	5.08	6.73	5.72	5.01	2.67	8.29	6.60	6.78	2.56	1.83	5.24	3.71	5.30	0.13	-0.64	1
1327	372	32	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	10.28	7.09	7.55	5.71	9.40	5.21	7.80	7.93	5.96	5.06	4.73	5.40	5.29	4.92	7.34	5.04	7.17	6.35	3.50	3.70	2.98	4.58	5.21	5.13	-2.39	1
483	372	32	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.59	8.26	6.80	5.54	6.45	1.71	8.21	7.18	4.40	6.21	3.59	0.69	7.21	8.25	5.67	6.58	11.00	6.69	0.90	4.72	-33.10	5.85	5.45	5.63	3.39	1
431	372	32	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.18	8.05	8.24	-28.48	3.55	6.12	9.77	6.47	9.64	6.34	7.28	2.89	4.44	4.35	9.29	7.22	2.08	9.14	4.27	4.25	1.32	6.18	4.40	5.56	3.15	1
1059	372	32	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.80	9.17	8.49	8.06	8.46	4.53	5.32	8.79	7.93	6.13	3.53	5.19	8.64	4.02	4.12	6.45	5.06	6.58	2.78	2.50	5.98	2.32	2.93	-4.70	1.83	1
556	372	32	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.36	7.89	6.18	8.22	4.28	4.79	6.83	6.66	8.74	2.09	5.91	0.70	7.22	7.44	5.99	7.54	7.96	9.66	-0.27	4.73	4.05	3.58	6.90	5.60	5.00	1
99	372	32	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.77	7.21	9.38	7.20	6.71	3.61	6.74	6.63	6.06	7.64	1.36	4.43	8.21	8.58	6.58	8.38	8.05	7.99	3.88	1.36	1.26	3.88	4.47	3.96	4.90	0
108	372	32	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	4.62	7.14	8.22	8.02	7.76	5.66	9.32	7.43	4.77	5.23	3.66	3.48	3.90	9.45	7.14	6.05	10.03	5.07	1.26	3.59	2.66	3.25	6.59	2.88	1.07	0
111	372	32	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.32	9.85	7.18	7.20	4.35	5.08	6.29	9.94	4.20	6.06	3.93	4.49	6.15	6.82	7.29	8.54	3.40	3.82	4.38	4.28	1.41	5.81	3.88	1.31	2.13	0
1452	372	32	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.20	8.26	7.52	9.39	9.99	5.19	10.01	9.01	7.64	6.77	2.13	6.01	6.14	6.49	7.46	5.79	6.34	8.26	5.84	2.02	0.68	2.91	3.91	4.60	2.42	0
1068	372	32	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.26	4.55	8.77	7.14	7.28	5.79	8.94	6.02	5.41	4.82	2.88	4.80	6.27	6.70	2.91	6.98	8.40	6.43	5.02	3.96	4.44	4.11	5.91	3.54	2.78	0
495	372	32	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.57	9.89	6.97	6.14	5.20	4.53	4.09	6.12	5.60	7.01	7.61	3.99	0.63	7.63	0.76	7.37	6.19	5.75	3.90	1.23	3.40	4.64	0.70	5.35	4.09	0
1324	372	32	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.34	4.06	6.27	8.09	7.05	6.37	6.00	9.28	7.68	5.72	2.98	5.56	8.53	4.81	5.53	4.37	8.48	5.62	2.94	2.34	3.82	6.23	3.35	4.23	3.87	0
739	372	32	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.94	6.51	9.37	5.11	4.40	6.11	7.76	6.32	3.96	4.90	3.70	4.34	2.84	7.39	8.90	6.83	9.79	5.87	5.52	1.76	4.46	3.27	2.77	2.96	0.17	0
1123	372	32	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	4.63	10.27	7.20	8.28	7.48	2.50	8.12	5.88	4.29	7.42	5.14	4.57	5.54	7.11	4.23	5.36	6.64	5.04	4.17	3.51	3.40	3.72	3.57	1.30	4.17	0
687	372	32	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.16	6.56	6.09	6.47	7.82	4.05	8.60	5.85	2.53	5.04	5.39	3.89	5.89	7.81	3.99	4.99	6.57	4.91	2.17	4.11	3.09	2.37	1.98	4.74	3.09	0
547	372	32	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.02	4.51	7.09	6.56	6.80	1.75	7.25	5.21	5.55	4.89	3.06	4.19	6.39	5.19	6.20	3.54	7.45	9.40	0.77	2.03	4.23	5.31	3.86	0.75	0.28	0

Model performance on valid data

Average & Std Deviation of the Significant rows:

	mean	std
mAP_valid_abs_values_26	89.623158	1.088802
mAP_valid_abs_values_21	89.357368	1.085571
mAP_valid_abs_values_19	89.085789	1.550263
mAP_valid_abs_values_22	89.069474	1.367997
mAP_valid_abs_values_20	89.046316	1.085737
mAP_valid_abs_values_25	88.888947	1.346130
mAP_valid_abs_values_12	88.786316	1.147559
mAP_valid_abs_values_11	88.416316	0.784830
mAP_valid_abs_values_10	88.139474	1.112372
mAP_valid_abs_values_6	87.949474	1.254793
mAP_valid_abs_values_13	87.906316	1.202443
mAP_valid_abs_values_16	87.740000	1.109920
mAP_valid_abs_values_18	87.481053	1.076455
mAP_valid_abs_values_15	87.450000	1.354732
mAP_valid_abs_values_14	87.040526	0.977488
mAP_valid_abs_values_17	86.977895	1.384628
mAP_valid_abs_values_3	86.885263	1.095893
mAP_valid_abs_values_8	86.877368	0.713737
mAP_valid_abs_values_23	86.757895	10.481356
mAP_valid_abs_values_4	86.663158	0.974765
mAP_valid_abs_values_7	86.481053	1.548870
mAP_valid_abs_values_2	86.401053	0.950777
mAP_valid_abs_values	86.276842	0.964135
mAP_valid_abs_values_9	85.042105	10.076121
mAP_valid_abs_values_5	84.387368	9.971664

Model performance on Test data

We then apply the hyperparameters to the test set and average the results.

test_data_1['average_map'] = test_data_1_sub.apply(np.mean, axis=1)

test_data_1.sort_values(by=['average_map'], ascending= False,inplace=True)

	mean	std
mAP_test_abs_values_3	94.345789	1.125543
mAP_test_abs_values_16	94.287895	1.039683
mAP_test_abs_values_7	94.236316	1.023796
mAP_test_abs_values_18	94.145263	0.857713
mAP_test_abs_values_17	94.100526	1.378427
mAP_test_abs_values_8	93.969474	1.159399
mAP_test_abs_values_2	93.910000	1.177474
mAP_test_abs_values_10	93.900000	0.908124
mAP_test_abs_values_13	93.899474	1.690419
mAP_test_abs_values_14	93.609474	1.573024
mAP_test_abs_values	93.507895	1.229479
mAP_test_abs_values_15	93.281579	1.288113
mAP_test_abs_values_12	93.134737	0.932258
average_map	92.748105	1.149440
mAP_test_abs_values_6	92.634737	0.814578
mAP_test_abs_values_11	92.588947	1.279613
mAP_test_abs_values_19	92.277895	1.149974
mAP_test_abs_values_26	92.025789	1.318582
mAP_test_abs_values_4	91.967368	8.027002
mAP_test_abs_values_20	91.865789	1.189521
mAP_test_abs_values_25	91.660000	2.404507
mAP_test_abs_values_9	91.295789	10.566407
mAP_test_abs_values_22	91.276316	7.862989
mAP_test_abs_values_5	90.944211	10.557963
mAP_test_abs_values_21	90.854737	8.062013
mAP_test_abs_values_23	88.982632	12.615593

Model performance on validation data (Mann-Whitney U Test)

Statistical Significance:

Using the global mean maximum row as the reference, we perform the right-tailed t-test to identify significant hyperparameters.

pvalues_mw = []
pvalues_real_mw = []

for _,index in enumerate(dt_valid_sub.index):

  _, p_value_mw = stats.mannwhitneyu(max_row[:-1], dt_valid_sub.loc[index][:-1], alternative='greater')#, method = 'asymptotic')
  pvalues_real_mw.append(p_value_mw)
  pvalues_mw.append((p_value_mw >= 0.05)*1)
  

dt_valid_sub_mw['pvalues_mw'] = pvalues_real_mw
dt_valid_sub_mw['sig_mw'] = pvalues_mw

dt_valid_sub_mw1 = dt_valid_sub_mw.query('sig_mw == 1')


max_indexx = dt_valid_sub_mw1.iloc[:, :-3].mean().idxmax()

#print(sum(pvalues),max_indexx)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_abs_values	mAP_valid_abs_values_2	mAP_valid_abs_values_3	mAP_valid_abs_values_4	mAP_valid_abs_values_5	mAP_valid_abs_values_6	mAP_valid_abs_values_7	mAP_valid_abs_values_8	mAP_valid_abs_values_9	mAP_valid_abs_values_10	mAP_valid_abs_values_11	mAP_valid_abs_values_12	mAP_valid_abs_values_13	mAP_valid_abs_values_14	mAP_valid_abs_values_15	mAP_valid_abs_values_16	mAP_valid_abs_values_17	mAP_valid_abs_values_18	mAP_valid_abs_values_19	mAP_valid_abs_values_20	mAP_valid_abs_values_21	mAP_valid_abs_values_22	mAP_valid_abs_values_23	mAP_valid_abs_values_25	mAP_valid_abs_values_26
1452	372	32	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	85.93	86.79	87.17	86.05	84.99	87.28	85.76	86.04	87.79	88.24	89.22	87.85	87.40	87.91	85.73	89.09	88.92	87.09	87.44	89.69	91.22	88.93	88.92	88.60	89.20
687	372	32	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	86.32	86.76	87.05	87.52	86.74	87.02	86.38	87.31	88.17	86.68	88.64	89.54	87.36	87.10	87.99	88.63	86.43	87.45	89.60	88.46	89.09	89.80	90.05	88.30	88.81
1059	372	32	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	86.70	85.32	87.35	86.52	86.28	87.35	87.73	86.71	86.79	87.70	89.10	88.22	86.50	86.82	87.90	88.35	87.90	87.53	90.55	88.27	87.58	89.38	90.06	89.91	89.77
1068	372	32	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	87.27	87.65	87.08	85.49	87.92	86.72	86.20	87.75	88.60	89.02	88.44	88.49	89.48	85.41	87.67	88.37	87.00	87.65	89.04	89.17	88.42	89.17	88.13	90.03	90.25
1071	372	32	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	87.21	85.68	88.49	87.98	89.55	86.99	87.26	87.18	87.88	87.80	88.27	86.99	87.97	86.66	87.00	86.67	86.82	87.74	87.50	88.64	89.37	89.69	91.14	89.71	87.49
99	372	32	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	86.47	87.08	84.36	86.55	86.57	88.08	87.45	86.88	87.08	86.79	89.07	88.21	86.33	85.30	88.03	86.91	86.80	87.80	89.13	89.74	90.60	89.02	89.72	88.91	89.06
111	372	32	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	84.54	84.88	87.87	86.70	88.55	87.58	87.46	86.24	89.87	88.57	88.20	89.83	89.44	87.26	87.25	86.37	87.98	89.71	87.20	87.60	90.87	86.13	88.69	88.61	89.97
495	372	32	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	85.53	86.15	85.32	86.37	83.24	89.48	90.21	88.14	87.93	88.02	86.47	89.64	90.62	86.92	91.48	87.75	87.88	87.74	88.88	91.25	89.24	88.83	90.97	87.81	89.87
1324	372	32	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	85.43	88.18	88.08	86.56	84.62	87.21	86.19	86.31	87.31	88.32	88.56	87.75	87.61	89.22	87.36	89.85	85.79	87.26	89.76	89.20	88.95	87.80	89.53	87.47	88.24
1123	372	32	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	88.99	85.29	87.42	87.70	85.80	90.65	86.24	88.33	86.26	86.88	87.69	88.38	88.70	87.21	88.69	86.83	87.12	88.26	88.04	88.31	89.53	90.34	90.14	89.97	89.32
1135	372	32	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	86.14	87.21	88.77	85.25	88.44	87.57	84.43	86.61	85.73	87.44	87.96	87.76	87.43	87.85	89.03	87.65	86.69	86.97	90.06	91.21	88.72	90.81	88.03	91.39	90.79
547	372	32	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	86.14	87.67	86.50	87.07	87.51	89.98	88.49	87.38	87.17	89.53	88.39	89.59	88.90	88.02	86.27	90.24	85.71	85.20	92.18	90.23	88.57	88.39	89.98	91.55	91.02

Size of the All data:  (98, 28)
Size of the Sig data:  (12, 33)

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params_mw1 = dt_valid.loc[dt_valid_sub_mw1.index][['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
optimal_params_mw = optimal_params_mw1.reset_index()
optimal_params_mw.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_mw1 = pd.merge(optimal_params_mw, dt_test, on=['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg'])
test_data_1_sub_mw = test_data_mw1[list(test_data_mw1.columns[test_data_mw1.columns.str.startswith('mAP_test')])]

Equivalent test data

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_abs_values	mAP_test_abs_values_2	mAP_test_abs_values_3	mAP_test_abs_values_4	mAP_test_abs_values_5	mAP_test_abs_values_6	mAP_test_abs_values_7	mAP_test_abs_values_8	mAP_test_abs_values_9	mAP_test_abs_values_10	mAP_test_abs_values_11	mAP_test_abs_values_12	mAP_test_abs_values_13	mAP_test_abs_values_14	mAP_test_abs_values_15	mAP_test_abs_values_16	mAP_test_abs_values_17	mAP_test_abs_values_18	mAP_test_abs_values_19	mAP_test_abs_values_20	mAP_test_abs_values_21	mAP_test_abs_values_22	mAP_test_abs_values_23	mAP_test_abs_values_25	mAP_test_abs_values_26
9	372	32	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	93.13	95.05	94.69	95.44	94.98	92.47	95.77	95.05	95.43	95.01	91.35	93.86	93.54	94.40	93.19	94.88	95.26	95.35	93.28	91.71	91.90	91.84	92.83	93.20	91.62
7	372	32	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	91.48	93.32	93.14	93.99	94.56	91.07	94.98	93.16	90.70	91.72	94.03	93.43	93.25	94.91	91.98	93.62	93.00	92.36	91.77	92.57	92.18	92.17	92.03	93.04	91.90
8	372	32	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	93.50	94.49	95.84	94.58	94.74	91.88	93.05	95.50	94.72	93.83	92.63	93.41	95.14	90.84	92.02	94.80	92.96	94.11	93.33	90.77	93.56	91.70	92.99	85.21	91.60
3	372	32	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	94.53	92.20	95.85	92.63	95.20	92.51	95.14	93.77	94.01	93.84	91.32	93.29	95.75	92.11	90.58	95.35	95.40	94.08	94.06	93.13	92.86	93.28	94.04	93.57	93.03
5	372	32	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	94.26	93.58	95.26	95.27	94.41	92.67	93.13	94.79	93.76	94.20	92.42	93.58	94.05	93.31	93.45	94.08	93.73	94.78	93.32	92.63	94.76	93.40	59.16	91.82	92.89
11	372	32	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	96.24	94.29	93.74	93.75	93.28	91.69	94.19	93.51	93.14	94.43	90.43	92.64	94.54	93.88	94.61	95.29	94.85	95.79	93.01	91.10	91.86	92.90	94.19	92.87	93.96
6	372	32	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	92.86	94.73	95.05	93.90	92.90	92.66	93.75	96.18	94.07	94.63	92.13	94.32	95.59	94.08	94.54	94.91	91.38	93.53	91.58	91.88	92.28	91.94	92.57	89.92	92.10
1	372	32	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	93.10	96.04	92.29	92.51	88.44	94.01	94.30	94.26	93.53	95.03	94.08	93.63	91.25	94.55	92.24	95.12	94.07	93.49	92.78	92.48	92.64	93.47	91.67	93.16	93.96
10	372	32	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	93.77	92.24	94.35	94.65	91.67	93.58	92.19	95.59	94.99	94.04	91.54	93.31	96.14	94.03	92.89	94.22	94.27	92.88	92.70	91.54	92.77	94.03	92.88	91.70	92.11
2	372	32	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	93.62	95.56	94.62	95.98	93.28	93.15	94.36	94.21	90.55	94.30	92.83	92.95	94.24	94.32	92.92	92.19	93.76	93.30	92.21	91.82	92.93	94.06	93.71	91.27	93.49
4	372	32	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	92.27	95.17	93.79	93.82	94.48	91.68	94.70	92.62	95.45	92.59	93.04	94.49	93.15	92.86	91.70	95.94	93.29	93.75	92.62	93.04	93.96	94.52	93.33	91.52	90.15
0	372	32	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	92.16	92.18	93.59	93.63	94.31	91.73	95.74	92.59	92.72	94.42	91.45	93.78	95.29	93.21	92.47	93.78	93.16	94.60	92.95	92.26	92.80	93.70	93.84	92.30	91.30

Size of the test data:  (12, 33)

Difference

diff_mw = dt_mw[['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
for i, cols in enumerate(zip(dt_mw.columns,test_data_mw1.columns)):   
   diff_mw[f'col_diff_{i}'] = (np.array(dt_mw[cols[0]]) - np.array(test_data_mw1[cols[1]]))*-1

# Function to count negatives in a row
def count_negatives(row):
    return sum(1 for value in row if isinstance(value, (int, float)) and value < 0)

diff_mw['negative_count'] = diff_mw.apply(count_negatives, axis=1)

diff_mw.sort_values(by =["negative_count"],inplace = True, ascending=False)


html_table = diff_mw.to_html(index=True)

# Wrap in a scrollable div
scrollable_table = f"""
<div style="height: 400px; width: 100%; overflow-x: auto; overflow-y: auto;">
    {html_table}
</div>
"""
# Display the scrollable table
display(HTML(scrollable_table))

key_values = ['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']
print('Are the keys of the valid and test dfs same?: ',dt_mw[key_values].equals(test_data_mw1[key_values]))

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	col_diff_0	col_diff_1	col_diff_2	col_diff_3	col_diff_4	col_diff_5	col_diff_6	col_diff_7	col_diff_8	col_diff_9	col_diff_10	col_diff_11	col_diff_12	col_diff_13	col_diff_14	col_diff_15	col_diff_16	col_diff_17	col_diff_18	col_diff_19	col_diff_20	col_diff_21	col_diff_22	col_diff_23	col_diff_24	col_diff_25	col_diff_26	col_diff_27	col_diff_28	col_diff_29	col_diff_30	col_diff_31	col_diff_32	negative_count
1059	372	32	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.80	9.17	8.49	8.06	8.46	4.53	5.32	8.79	7.93	6.13	3.53	5.19	8.64	4.02	4.12	6.45	5.06	6.58	2.78	2.50	5.98	2.32	2.93	-4.70	1.83	1
1071	372	32	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.05	7.90	6.77	7.29	4.86	5.68	5.87	7.61	5.88	6.40	4.15	6.59	6.08	6.65	6.45	7.41	6.91	7.04	5.82	3.99	5.39	3.71	-31.98	2.11	5.40	1
1135	372	32	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.13	7.96	5.02	8.57	6.04	4.11	10.27	6.01	9.72	5.15	5.08	6.73	5.72	5.01	2.67	8.29	6.60	6.78	2.56	1.83	5.24	3.71	5.30	0.13	-0.64	1
1452	372	32	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.20	8.26	7.52	9.39	9.99	5.19	10.01	9.01	7.64	6.77	2.13	6.01	6.14	6.49	7.46	5.79	6.34	8.26	5.84	2.02	0.68	2.91	3.91	4.60	2.42	0
687	372	32	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.16	6.56	6.09	6.47	7.82	4.05	8.60	5.85	2.53	5.04	5.39	3.89	5.89	7.81	3.99	4.99	6.57	4.91	2.17	4.11	3.09	2.37	1.98	4.74	3.09	0
1068	372	32	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.26	4.55	8.77	7.14	7.28	5.79	8.94	6.02	5.41	4.82	2.88	4.80	6.27	6.70	2.91	6.98	8.40	6.43	5.02	3.96	4.44	4.11	5.91	3.54	2.78	0
99	372	32	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.77	7.21	9.38	7.20	6.71	3.61	6.74	6.63	6.06	7.64	1.36	4.43	8.21	8.58	6.58	8.38	8.05	7.99	3.88	1.36	1.26	3.88	4.47	3.96	4.90	0
111	372	32	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.32	9.85	7.18	7.20	4.35	5.08	6.29	9.94	4.20	6.06	3.93	4.49	6.15	6.82	7.29	8.54	3.40	3.82	4.38	4.28	1.41	5.81	3.88	1.31	2.13	0
495	372	32	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.57	9.89	6.97	6.14	5.20	4.53	4.09	6.12	5.60	7.01	7.61	3.99	0.63	7.63	0.76	7.37	6.19	5.75	3.90	1.23	3.40	4.64	0.70	5.35	4.09	0
1324	372	32	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.34	4.06	6.27	8.09	7.05	6.37	6.00	9.28	7.68	5.72	2.98	5.56	8.53	4.81	5.53	4.37	8.48	5.62	2.94	2.34	3.82	6.23	3.35	4.23	3.87	0
1123	372	32	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	4.63	10.27	7.20	8.28	7.48	2.50	8.12	5.88	4.29	7.42	5.14	4.57	5.54	7.11	4.23	5.36	6.64	5.04	4.17	3.51	3.40	3.72	3.57	1.30	4.17	0
547	372	32	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.02	4.51	7.09	6.56	6.80	1.75	7.25	5.21	5.55	4.89	3.06	4.19	6.39	5.19	6.20	3.54	7.45	9.40	0.77	2.03	4.23	5.31	3.86	0.75	0.28	0

Are the keys of the valid and test dfs same?:  False

Model performance on valid data

Average & Std Deviation of the Significant rows:

	mean	std
mAP_valid_abs_values_23	89.613333	1.001457
mAP_valid_abs_values_26	89.482500	1.017297
mAP_valid_abs_values_25	89.355000	1.300472
mAP_valid_abs_values_21	89.346667	1.072180
mAP_valid_abs_values_20	89.314167	1.158286
mAP_valid_abs_values_19	89.115000	1.455257
mAP_valid_abs_values_22	89.024167	1.222415
mAP_valid_abs_values_12	88.520833	0.920607
mAP_valid_abs_values_11	88.334167	0.746123
mAP_valid_abs_values_13	88.145000	1.295534
mAP_valid_abs_values_16	88.059167	1.258075
mAP_valid_abs_values_6	87.992500	1.303290
mAP_valid_abs_values_10	87.915833	0.888620
mAP_valid_abs_values_15	87.866667	1.466966
mAP_valid_abs_values_9	87.548333	1.093666
mAP_valid_abs_values_18	87.533333	1.021410
mAP_valid_abs_values_14	87.140000	1.086211
mAP_valid_abs_values_3	87.121667	1.260497
mAP_valid_abs_values_17	87.086667	0.942775
mAP_valid_abs_values_8	87.073333	0.742555
mAP_valid_abs_values_7	86.983333	1.475129
mAP_valid_abs_values_5	86.684167	1.832561
mAP_valid_abs_values_4	86.646667	0.830447
mAP_valid_abs_values_2	86.555000	1.076421
mAP_valid_abs_values	86.389167	1.117403

Model performance on Test data

We then apply the hyperparameters to the test set and average the results.

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params = dt_valid.loc[dt_valid_sub_mw1.index].iloc[:, :6].reset_index()
optimal_params.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_1 = pd.merge(optimal_params, dt_test, how='left')
test_data_1_sub = test_data_1[list(test_data_1.columns[test_data_1.columns.str.startswith('mAP_test')])]
test_data_1['average_map'] = test_data_1_sub.apply(np.mean, axis=1)

test_data_1.sort_values(by=['average_map'], ascending= False,inplace=True)

	mean	std
mAP_test_abs_values_3	94.380208	1.060424
mAP_test_abs_values_17	94.047917	1.308239
mAP_test_abs_values_13	94.010000	1.274458
mAP_test_abs_values_10	93.969375	0.840849
mAP_test_abs_values_4	93.921875	1.318254
mAP_test_abs_values_8	93.868750	1.137492
mAP_test_abs_values_14	93.770833	1.370117
mAP_test_abs_values	93.608958	1.126441
mAP_test_abs_values_18	93.466042	1.417141
mAP_test_abs_values_5	93.438333	1.878001
mAP_test_abs_values_12	93.129167	0.857018
mAP_test_abs_values_9	93.090833	6.731690
mAP_test_abs_values_15	92.981250	1.145863
average_map	92.800125	1.150285
mAP_test_abs_values_19	92.611667	1.244397
mAP_test_abs_values_11	92.546875	1.062498
mAP_test_abs_values_7	92.255833	9.350205
mAP_test_abs_values_6	92.234375	0.811699
mAP_test_abs_values_16	92.146875	9.339141
mAP_test_abs_values_20	92.142083	1.087787
mAP_test_abs_values_22	92.139583	5.044086
mAP_test_abs_values_26	91.914375	1.499664
mAP_test_abs_values_2	91.724375	8.675925
mAP_test_abs_values_25	91.643125	2.238106
mAP_test_abs_values_23	91.098958	8.427058
mAP_test_abs_values_21	89.861458	9.058537

Summary using radar plot

def extract_number(text):
    if isinstance(text, str):
        matches = re.findall(r'\d+', text)
        return int(matches[0]) if matches else 1
    return 1

res_valid1['id'] = res_valid1.index.to_series().apply(extract_number)
res_test1['id'] = res_test1.index.to_series().apply(extract_number)



res_comb = pd.concat([res_valid1,res_test1])
index_series = res_comb.index.to_series()

res_comb['type'] = np.where(
    index_series.str.contains('valid', case=False), 'valid',
    np.where(index_series.str.contains('test', case=False), 'test', 'unknown')
)
res_comb = res_comb.query("type !='unknown'")

res_comb =  res_comb.reset_index(drop=True)


res_test1 = res_test1.sort_values(by=['id']).reset_index().query("index !='average_map'")
data_range1 = np.array(list(res_test1['mean']) + list(res_valid1['mean']))
categories = [str(i) for i in range(1,26)]
fig = go.Figure()

# Valid
fig.add_trace(go.Scatterpolar(
       r=list(res_test1['mean']),
      theta=categories,
      #fill='toself',
      name='Test'
))

# Test
fig.add_trace(go.Scatterpolar(
      r=list(res_valid1['mean']),
      theta=categories,
      #fill='toself',
      name='Valid'
))

# Customization of chart
fig.update_layout(
  title={
      'text': "T-Test",
        'xanchor': 'center',
      'yanchor': 'top'
  },
  polar=dict(
    radialaxis=dict(
      visible=True,
      range=[round(data_range1.min()), round(data_range1.max()) + 1]
    )),
  showlegend=True
)

fig.show()

##############


res_valid2['id'] = res_valid2.index.to_series().apply(extract_number)
res_test2['id'] = res_test2.index.to_series().apply(extract_number)



res_comb = pd.concat([res_valid2,res_test2])
index_series = res_comb.index.to_series()

res_comb['type'] = np.where(
    index_series.str.contains('valid', case=False), 'valid',
    np.where(index_series.str.contains('test', case=False), 'test', 'unknown')
)
res_comb = res_comb.query("type !='unknown'")

res_comb =  res_comb.reset_index(drop=True)


res_test2 = res_test2.sort_values(by=['id']).reset_index().query("index !='average_map'")

data_range2 = np.array(list(res_test2['mean']) + list(res_valid2['mean']))

categories = [str(i) for i in range(1,26)]
fig = go.Figure()

# Valid
fig.add_trace(go.Scatterpolar(
       r=list(res_test2['mean']),
      theta=categories,
      #fill='toself',
      name='Test'
))

# Test
fig.add_trace(go.Scatterpolar(
      r=list(res_valid2['mean']),
      theta=categories,
      #fill='toself',
      name='Valid'
))

# Customization of chart
fig.update_layout(
  title={
      'text': "MU-Test",
         'xanchor': 'center',
      'yanchor': 'top'
  },
  polar=dict(
    radialaxis=dict(
      visible=True,
      range=[round(data_range2.min()), round(data_range2.max()) + 1]
    )),
  showlegend=True
)

fig.show()

Experiment 2 Results

• In these experimental results, the thresholding is based on a fixed value of 0. This decision is informed by the fact that the binary-like hash values are symmetrically distributed between -1 and 1.

Validation Data mAP Results Preview:

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_zero	mAP_valid_zero_2	mAP_valid_zero_3	mAP_valid_zero_4	mAP_valid_zero_5	mAP_valid_zero_6	mAP_valid_zero_7	mAP_valid_zero_8	mAP_valid_zero_9	mAP_valid_zero_10	mAP_valid_zero_11	mAP_valid_zero_12	mAP_valid_zero_13	mAP_valid_zero_14	mAP_valid_zero_15	mAP_valid_zero_16	mAP_valid_zero_17	mAP_valid_zero_18	mAP_valid_zero_19	mAP_valid_zero_20	mAP_valid_zero_21	mAP_valid_zero_22	mAP_valid_zero_23	mAP_valid_zero_25	mAP_valid_zero_26
32	372	32	0.1	16	0.00100	24	0.000000e+00	0.000000e+00	84.31	84.04	85.93	87.19	86.39	84.06	84.62	84.74	86.46	85.18	88.63	86.20	86.94	84.54	84.80	85.63	88.12	87.64	87.13	87.82	84.72	87.71	89.13	87.89	86.72
35	372	32	0.1	16	0.00100	24	0.000000e+00	1.000000e-08	84.35	85.16	86.51	84.89	85.95	85.98	86.27	85.65	85.08	85.31	89.38	87.45	85.32	83.38	86.84	87.45	85.82	43.72	88.81	84.95	89.39	85.94	86.46	89.80	88.16
44	372	32	0.1	16	0.00100	24	1.000000e-08	0.000000e+00	85.96	85.59	85.49	85.19	83.88	85.89	86.52	88.78	85.56	85.36	87.82	87.53	85.74	85.68	87.69	86.41	86.65	87.16	88.21	87.96	90.19	89.02	89.74	87.93	89.19
47	372	32	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	85.87	86.44	85.19	86.03	84.77	90.44	82.24	86.48	85.18	85.96	87.39	85.60	85.18	85.62	87.64	86.41	86.86	87.66	88.48	86.82	88.17	87.88	85.48	87.90	86.77
96	372	32	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	84.30	86.50	85.12	83.83	84.80	87.50	43.72	88.58	86.78	87.46	89.44	89.08	86.27	88.21	88.22	43.72	86.57	86.72	89.45	88.68	89.03	89.22	88.34	87.33	86.57
99	372	32	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	86.42	86.66	84.35	86.37	85.86	88.07	87.45	86.78	86.34	86.78	87.78	88.20	86.33	85.27	87.98	86.91	86.51	87.20	89.13	89.74	90.59	88.83	89.70	88.85	88.70
108	372	32	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	87.14	86.12	86.93	86.42	86.69	87.45	85.50	85.75	89.14	86.49	88.28	88.49	88.09	86.37	86.88	86.41	86.63	88.65	88.82	87.98	88.90	87.16	87.13	89.85	89.99
111	372	32	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	84.30	84.87	87.86	86.25	88.54	87.57	87.32	85.68	89.37	88.30	88.20	89.58	89.44	86.68	87.06	86.35	86.01	88.71	87.18	87.60	90.85	85.89	88.69	88.60	89.95
160	372	32	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	86.51	86.54	84.32	84.84	84.95	86.69	84.18	86.53	86.00	85.33	86.99	86.48	86.18	87.75	85.43	79.53	86.74	83.66	88.30	90.34	87.17	87.96	85.78	88.02	88.85
163	372	32	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	85.36	86.78	85.53	85.62	78.22	86.47	85.32	86.26	88.23	87.51	87.70	88.55	87.08	87.57	87.64	84.70	86.11	85.68	89.07	87.63	90.72	86.94	86.64	88.52	90.09
172	372	32	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	84.38	85.26	85.22	86.82	87.16	88.07	86.87	43.72	87.42	87.28	86.24	85.46	86.88	43.72	84.60	88.26	86.92	86.24	89.14	87.61	86.63	87.44	43.72	88.81	86.80
175	372	32	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	83.16	88.60	85.04	85.52	86.39	86.92	84.30	88.31	88.16	85.55	88.13	89.59	89.48	87.57	85.25	88.03	86.30	85.66	88.69	88.61	89.02	88.05	88.19	87.98	87.99
224	372	32	0.1	16	0.00100	36	0.000000e+00	0.000000e+00	82.63	84.16	86.58	85.32	82.73	87.94	83.67	86.38	85.54	87.01	88.05	88.94	85.13	86.99	86.58	87.13	87.18	85.80	87.87	88.36	88.42	87.72	88.35	86.93	87.54
227	372	32	0.1	16	0.00100	36	0.000000e+00	1.000000e-08	86.98	85.44	86.56	87.65	84.46	87.95	86.15	86.01	84.96	85.15	86.97	86.78	85.99	85.83	89.61	86.42	85.60	86.68	89.03	87.74	86.16	86.16	86.90	87.71	87.46
236	372	32	0.1	16	0.00100	36	1.000000e-08	0.000000e+00	87.45	85.17	85.84	83.80	86.43	85.65	84.65	86.26	87.07	83.87	88.27	85.98	85.85	88.32	85.86	84.88	84.81	86.37	88.41	88.38	90.44	89.61	88.18	89.90	87.83
239	372	32	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	85.42	85.42	85.54	85.23	87.12	86.68	83.04	84.49	83.50	86.21	87.63	91.61	85.86	86.00	86.32	87.59	86.93	86.45	88.45	87.61	89.61	87.18	88.67	88.58	88.06
288	372	32	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	86.03	83.88	84.60	86.51	85.45	85.51	86.20	88.17	88.73	85.81	86.22	88.38	83.77	87.31	85.14	86.11	43.72	84.85	43.72	86.53	88.60	86.69	88.65	88.44	88.58
291	372	32	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	84.52	86.89	86.53	83.71	85.18	85.43	84.76	85.28	87.50	85.92	88.09	87.30	88.80	85.49	87.25	87.97	86.67	85.56	86.50	87.52	88.74	88.43	88.16	89.34	87.53
300	372	32	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	86.67	86.60	86.23	83.28	86.42	86.37	86.54	89.26	85.25	85.59	87.28	87.30	87.86	84.83	86.27	87.73	87.31	85.83	89.32	88.78	88.10	88.29	43.72	87.05	87.93
303	372	32	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	85.05	86.07	86.60	83.70	86.98	88.62	86.08	87.38	86.42	87.48	88.10	87.68	87.27	87.08	86.97	86.42	88.79	87.14	88.09	87.75	88.16	86.22	89.38	89.43	87.77
352	372	32	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	85.75	83.50	85.70	84.00	85.98	86.72	81.62	87.33	84.24	87.65	87.25	88.11	87.50	88.50	87.18	87.58	85.98	87.41	86.69	88.82	89.76	87.52	87.09	87.57	86.67
355	372	32	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	84.90	84.63	84.62	84.80	86.51	86.73	87.04	86.10	87.08	87.07	86.10	87.27	86.46	85.69	85.42	87.32	86.48	88.19	87.28	88.50	88.24	87.36	89.09	87.09	88.38
364	372	32	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	85.79	88.77	86.03	84.09	86.58	86.27	86.72	88.48	86.35	88.62	87.47	87.08	85.71	84.43	89.09	86.81	88.67	87.24	88.92	88.40	89.00	89.86	88.20	81.36	88.05
367	372	32	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	86.33	86.86	85.54	89.15	86.08	86.74	85.40	87.35	88.36	86.29	88.20	86.72	88.51	85.72	86.91	87.22	86.80	86.62	89.00	87.47	89.87	88.35	86.95	88.72	87.41
416	372	32	0.1	16	0.00100	48	0.000000e+00	0.000000e+00	85.48	84.98	84.76	82.01	85.13	85.95	85.12	86.39	85.52	88.34	84.46	85.70	87.90	87.41	85.01	87.78	87.36	87.55	87.79	89.87	86.88	88.46	83.26	86.52	89.29
419	372	32	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	85.29	84.90	85.67	85.85	84.71	85.33	85.43	86.65	86.64	86.38	87.41	86.15	87.20	86.34	89.07	87.52	85.25	87.17	88.29	87.00	88.35	88.87	90.13	87.53	89.84
428	372	32	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	86.23	89.22	85.11	85.17	86.29	88.74	85.41	85.19	84.92	88.14	87.76	86.73	87.89	86.42	86.99	87.29	87.26	85.14	89.12	88.20	89.69	89.38	88.87	89.75	86.76
431	372	32	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	85.72	85.64	86.08	86.95	84.17	87.46	84.13	85.69	85.26	87.48	88.10	89.88	87.50	85.01	85.96	86.58	90.29	85.16	89.55	87.85	90.40	88.13	43.72	87.04	88.91
480	372	32	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	86.17	84.98	86.86	86.45	86.11	86.21	84.72	86.26	86.57	87.74	89.45	88.88	86.63	85.74	86.97	87.84	85.60	88.13	89.07	89.96	87.34	92.58	88.91	88.19	89.47
483	372	32	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	85.38	86.74	87.22	87.06	87.69	89.67	83.53	86.28	43.72	87.40	89.52	89.77	86.61	87.14	87.93	87.40	85.41	87.50	90.02	87.86	90.82	88.34	87.78	86.14	89.74
492	372	32	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	84.67	86.84	85.39	86.52	84.09	84.87	84.68	84.53	86.24	85.67	87.12	90.33	86.83	87.08	86.92	87.46	88.69	88.75	88.08	88.08	89.78	88.09	89.51	88.95	88.92
495	372	32	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	85.36	85.76	85.16	86.10	83.23	88.14	89.99	87.93	87.86	88.02	86.47	89.21	90.60	86.89	89.18	87.73	87.43	87.74	88.73	91.18	89.22	88.78	90.97	87.81	89.84
544	372	32	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	85.71	84.84	84.79	85.10	85.47	87.00	83.61	85.53	86.67	85.60	86.32	86.58	88.25	84.91	85.71	88.05	82.84	88.30	92.18	87.97	89.53	90.33	86.33	88.23	88.53
547	372	32	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	86.14	87.66	86.50	87.01	87.47	89.74	87.98	87.14	87.17	89.51	88.39	89.23	88.88	87.79	86.27	90.24	85.51	84.89	92.18	89.88	88.57	88.38	89.94	91.49	90.34
556	372	32	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	85.78	85.43	85.27	84.18	87.69	86.67	87.23	86.48	85.79	91.00	88.08	91.82	87.31	87.12	86.77	87.10	85.69	85.24	91.76	87.93	87.94	89.72	87.41	88.59	87.66
559	372	32	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	85.56	84.04	87.13	86.30	84.11	87.30	84.34	87.79	84.94	83.43	89.31	87.27	87.17	85.45	89.60	87.13	85.04	86.15	89.72	90.16	88.81	89.15	88.30	89.00	88.98
608	372	32	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	83.47	83.33	83.83	84.09	81.92	84.58	43.72	85.99	84.26	86.35	86.24	86.92	87.10	86.20	86.97	85.16	87.05	84.55	86.10	87.27	87.01	88.29	87.91	87.96	87.93
611	372	32	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	86.29	82.47	87.03	85.67	85.82	87.28	84.15	86.58	85.96	84.21	88.46	88.77	88.27	86.69	85.78	86.15	84.13	86.70	87.59	86.96	88.50	87.46	88.40	86.39	85.55
620	372	32	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	85.02	85.36	88.91	85.07	85.83	86.21	84.79	89.05	86.42	84.01	87.94	87.18	86.08	88.29	87.53	87.87	89.40	84.82	86.30	87.36	88.35	88.56	87.77	87.34	88.30
623	372	32	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	86.89	89.90	87.67	85.29	84.82	89.14	84.84	86.13	84.68	84.64	87.65	87.63	87.84	85.62	87.63	85.27	87.53	86.15	87.35	88.39	87.64	87.31	89.85	88.26	89.54
672	372	32	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	83.87	86.38	84.74	83.80	83.35	86.31	84.60	87.18	86.02	86.93	87.59	86.73	85.22	84.17	85.50	84.68	86.42	87.61	87.56	87.58	86.33	88.39	87.48	87.66	88.39
675	372	32	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	84.86	88.10	86.01	84.19	85.93	85.72	87.07	86.90	85.08	87.24	87.44	87.21	87.15	84.13	89.21	85.99	87.07	86.01	88.50	88.65	90.26	88.30	88.32	88.64	88.10
684	372	32	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	86.90	89.16	86.60	84.37	88.12	87.13	85.50	87.29	88.10	86.69	87.24	86.87	85.58	87.42	86.58	87.46	85.88	85.49	86.36	85.05	89.42	88.86	89.37	87.00	89.29
687	372	32	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	86.16	86.66	87.05	86.72	86.68	86.93	86.20	86.77	88.17	86.60	88.53	89.26	86.85	86.31	87.84	87.96	86.16	87.39	89.55	88.46	87.63	89.79	90.02	88.27	88.81
736	372	32	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	85.20	53.60	86.12	87.29	82.02	88.01	83.69	86.58	87.42	88.36	86.51	87.81	87.78	88.55	85.02	87.82	83.50	86.91	86.41	89.09	88.29	89.72	89.80	87.55	87.84
739	372	32	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	86.08	86.03	85.85	88.05	43.72	87.04	86.76	86.55	87.97	87.70	87.19	87.72	86.59	87.14	85.52	86.54	84.93	88.22	85.70	88.45	89.70	88.59	88.98	86.52	89.49
748	372	32	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	86.26	86.21	83.40	86.87	85.25	86.34	85.05	84.93	84.35	86.49	86.72	88.49	85.09	85.93	87.53	88.11	85.29	85.91	88.24	89.85	87.23	89.11	88.56	89.69	87.04
751	372	32	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	85.92	87.19	84.87	86.93	85.30	85.99	87.39	86.97	87.29	85.09	88.60	87.79	87.40	88.61	87.39	86.89	88.18	85.63	86.98	89.86	89.17	88.81	88.68	89.74	87.53
800	372	32	0.1	16	0.00010	36	0.000000e+00	0.000000e+00	86.97	86.37	85.12	89.55	88.55	85.02	84.69	84.77	84.39	86.52	85.15	88.73	87.53	87.27	86.85	86.76	85.36	86.73	90.44	87.61	88.12	88.19	88.80	86.75	86.19
803	372	32	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	84.66	86.28	85.76	85.55	86.61	87.54	85.29	85.27	85.34	85.38	87.24	87.45	87.45	83.55	86.92	86.48	86.69	88.49	87.98	88.95	89.01	86.54	88.06	87.63	88.07
812	372	32	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	85.34	85.51	85.36	89.40	85.70	86.87	83.17	87.52	86.99	86.44	86.52	88.46	84.92	86.99	84.34	86.40	86.22	86.50	87.96	88.75	88.03	86.09	88.44	89.21	86.52
815	372	32	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	83.33	86.70	85.90	87.40	83.70	84.84	84.49	85.11	86.02	86.38	87.57	88.58	87.36	87.18	86.19	87.59	83.24	86.95	87.96	88.00	88.35	86.96	87.93	87.05	88.21
864	372	32	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	84.65	86.51	85.95	85.17	86.22	86.78	85.09	86.27	43.72	87.02	87.74	89.55	86.32	86.61	86.48	84.78	86.36	85.92	88.28	88.41	88.60	88.68	85.46	88.39	89.26
867	372	32	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	86.61	87.05	86.85	87.45	86.28	85.70	85.79	85.84	87.41	84.99	90.08	88.88	85.84	87.81	87.34	86.06	87.96	87.43	89.33	86.30	89.09	88.83	90.59	87.90	87.00
876	372	32	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	87.84	90.38	86.13	85.61	85.27	87.08	85.01	86.81	87.33	84.99	87.00	86.59	85.98	87.18	87.72	89.51	85.72	86.24	86.73	89.31	90.28	86.94	89.05	88.65	87.14
879	372	32	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	87.75	86.15	86.67	87.01	86.22	87.23	86.09	86.79	85.99	85.51	86.44	87.69	85.73	85.90	85.94	85.75	87.12	88.82	88.67	89.02	87.99	88.34	88.07	88.62	87.38
928	372	32	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	84.82	89.97	87.20	85.05	86.09	86.81	82.78	85.86	87.31	88.62	87.30	87.88	85.61	89.76	87.51	85.93	86.13	85.94	86.73	87.35	88.64	87.00	88.84	88.56	83.35
931	372	32	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	84.46	86.52	85.21	84.03	86.37	87.93	85.15	84.52	86.89	86.40	88.62	88.15	84.38	85.85	90.04	87.96	86.23	87.68	86.36	88.56	88.72	89.93	89.38	64.92	89.00
940	372	32	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	83.25	86.75	87.36	86.79	85.60	87.40	85.76	86.08	84.24	84.46	88.56	87.08	88.40	87.75	86.34	87.05	85.03	85.97	87.80	87.85	87.37	88.07	87.77	89.11	88.88
943	372	32	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	86.02	87.71	84.12	87.17	86.24	86.92	85.57	87.44	87.62	85.62	87.03	86.29	86.87	86.42	87.02	85.91	86.43	88.13	88.16	88.61	88.22	88.50	89.42	81.55	86.97
992	372	32	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	83.68	86.19	85.59	83.24	84.78	84.33	85.36	85.27	89.53	86.03	88.86	88.30	87.64	87.63	83.13	87.54	88.42	86.44	88.92	88.90	89.81	89.01	88.63	88.90	82.12
995	372	32	0.1	16	0.00010	48	0.000000e+00	1.000000e-08	86.20	84.40	85.58	84.93	88.62	84.51	86.03	86.43	86.45	84.77	87.25	90.02	86.19	88.22	87.84	86.45	87.74	85.73	86.73	88.37	88.21	85.48	87.95	88.57	87.68
1004	372	32	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	83.89	85.26	89.23	85.75	86.08	86.40	84.98	85.72	85.68	87.74	88.44	83.82	84.20	86.97	86.88	89.54	86.63	86.70	87.15	87.72	88.56	88.66	87.58	87.80	89.16
1007	372	32	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	84.88	85.02	85.52	84.62	89.89	87.49	84.08	88.50	85.73	84.25	85.58	87.96	88.06	87.47	87.85	87.25	88.76	85.00	84.79	87.61	89.84	89.64	88.44	89.44	89.43
1056	372	32	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	86.16	91.52	85.59	84.31	83.82	86.47	85.24	87.60	87.15	84.54	88.29	87.74	86.37	87.66	86.58	87.66	85.14	86.43	88.98	86.84	90.96	89.38	88.02	90.12	89.62
1059	372	32	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	86.63	85.26	87.13	84.91	86.23	87.25	87.50	86.35	86.16	87.70	88.94	88.08	86.48	86.69	86.20	88.35	87.90	87.53	90.55	88.22	87.47	89.38	89.85	89.09	89.53
1068	372	32	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	87.26	87.64	86.94	85.23	87.37	86.72	86.13	87.41	88.48	87.73	88.42	88.20	89.48	85.12	87.28	88.36	86.92	87.45	89.00	89.17	88.41	89.15	88.13	89.70	90.25
1071	372	32	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	85.91	85.36	88.31	87.63	89.53	86.65	87.26	86.69	87.87	87.72	87.56	86.75	87.83	86.44	86.72	86.26	86.41	87.73	87.01	88.64	89.37	88.74	90.47	89.69	87.35
1120	372	32	0.1	64	0.00010	48	0.000000e+00	0.000000e+00	86.19	86.18	85.06	83.83	87.93	88.76	87.57	88.44	86.88	87.75	89.06	87.24	87.85	85.46	87.93	85.92	87.74	84.71	87.77	90.74	87.55	89.88	88.11	89.27	87.23
1123	372	32	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	88.22	85.28	87.40	86.97	85.51	90.35	86.11	88.33	86.26	86.83	87.45	88.38	88.58	87.14	88.45	86.83	86.75	88.04	88.04	88.06	89.53	90.14	90.03	89.96	89.31
1132	372	32	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	82.33	86.92	85.73	84.39	84.07	87.40	84.74	86.55	87.42	86.68	86.94	90.14	85.79	85.00	88.59	88.54	85.37	85.87	87.63	92.32	89.65	89.59	88.90	87.15	86.09
1135	372	32	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	86.14	86.91	88.76	84.75	88.43	87.26	84.35	86.14	85.73	87.43	87.71	87.63	87.28	87.85	88.83	87.53	86.64	86.60	89.99	91.21	88.72	90.77	87.78	91.36	90.53
1184	372	32	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	83.78	86.92	84.71	88.14	83.79	85.93	84.15	86.32	82.74	85.65	85.79	86.25	84.60	88.97	88.61	87.03	85.24	84.43	87.33	86.20	89.14	87.84	87.67	86.04	87.69
1187	372	32	0.1	16	0.00001	24	0.000000e+00	1.000000e-08	85.49	88.50	85.77	87.67	84.47	86.94	84.97	86.22	87.33	85.74	86.11	88.82	89.09	86.66	88.02	87.47	85.48	86.47	86.78	88.27	86.71	86.91	87.09	89.11	88.54
1196	372	32	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	84.51	83.13	83.22	83.00	86.70	85.83	83.70	85.33	84.92	85.90	87.90	87.68	89.53	86.23	88.67	86.09	87.10	87.11	88.69	90.33	87.14	88.83	87.98	88.91	89.26
1199	372	32	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	84.68	85.09	85.98	83.78	85.38	86.04	86.16	86.68	87.01	85.90	88.34	88.39	85.99	86.15	85.77	87.09	88.95	87.93	84.52	87.29	89.01	87.74	87.56	89.80	88.30
1248	372	32	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	84.47	84.99	85.53	83.56	86.32	87.51	86.40	86.31	85.92	84.93	88.01	87.87	88.54	86.17	86.20	86.88	84.21	85.56	90.51	88.02	90.00	87.98	87.57	86.97	88.08
1251	372	32	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	85.95	86.31	86.34	86.27	86.44	87.69	86.29	85.23	86.27	86.17	88.82	88.72	84.89	86.09	87.65	85.75	85.99	87.97	88.98	88.73	91.12	87.57	87.65	89.23	88.57
1260	372	32	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	86.85	86.02	87.37	86.00	85.23	86.41	85.80	85.82	86.05	87.21	88.02	89.09	86.12	83.73	89.07	86.10	85.64	87.80	88.80	87.80	88.09	87.98	88.43	88.17	87.62
1263	372	32	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	85.60	87.75	86.79	86.91	86.29	86.94	86.65	86.77	87.18	86.86	88.90	88.84	87.97	85.30	85.15	86.74	87.58	86.82	87.66	88.09	88.67	90.38	87.03	89.88	88.07
1312	372	32	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	83.56	84.43	85.96	85.39	87.16	85.32	87.49	88.68	84.70	85.93	86.29	87.64	86.63	86.64	87.60	87.85	87.56	85.64	89.18	90.21	88.59	73.19	85.33	89.23	89.14
1315	372	32	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	84.94	85.99	86.59	84.59	86.37	88.25	86.67	86.55	86.34	87.35	86.90	87.82	85.54	86.79	85.67	86.64	87.13	87.15	86.58	89.25	89.78	87.86	88.99	89.18	87.56
1324	372	32	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	84.79	88.02	88.08	86.31	84.39	86.73	85.75	86.29	87.30	87.91	88.32	87.74	87.61	88.24	87.36	89.60	85.69	87.07	89.42	89.20	88.88	87.71	89.25	87.47	88.23
1327	372	32	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	85.33	86.59	86.04	86.11	85.20	87.93	86.65	87.54	85.82	90.05	88.39	87.46	88.11	87.05	85.99	87.61	88.35	86.87	87.54	88.55	89.82	89.05	88.02	86.67	91.92
1376	372	32	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	85.67	85.51	87.99	84.25	85.08	87.55	84.22	87.03	86.95	86.87	87.05	88.02	86.44	84.60	86.56	86.31	87.32	85.36	85.45	86.70	87.70	86.12	87.75	88.94	86.67
1379	372	32	0.1	16	0.00001	36	0.000000e+00	1.000000e-08	83.68	85.16	87.50	83.94	85.56	85.18	84.74	86.03	83.48	85.26	87.62	90.89	85.94	88.14	84.40	88.05	84.87	87.40	87.80	88.24	86.82	85.71	85.05	87.92	88.14
1388	372	32	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	86.12	85.48	86.63	85.41	87.29	85.74	84.37	86.15	85.71	87.05	88.18	87.37	85.56	87.30	88.23	88.60	87.67	85.25	89.34	87.61	88.59	89.48	86.44	87.13	88.01
1391	372	32	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	85.31	88.10	84.84	85.52	84.37	85.39	84.14	87.23	83.62	86.86	87.38	87.01	88.14	86.16	87.13	87.66	85.64	84.52	89.90	88.10	86.80	89.24	88.85	86.33	87.35
1440	372	32	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	85.65	88.95	84.75	87.08	87.62	85.67	85.51	86.65	89.14	85.32	86.98	86.87	86.62	86.51	86.12	87.86	87.75	86.29	86.91	86.82	88.76	89.07	88.87	88.49	87.41
1443	372	32	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	87.51	85.18	87.25	86.33	84.80	84.80	86.19	85.42	87.29	85.36	86.92	86.65	87.12	85.38	87.38	86.48	87.04	86.72	88.67	86.79	87.23	86.98	88.22	88.21	88.43
1452	372	32	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	85.91	86.72	87.17	85.85	84.96	87.28	85.76	85.79	87.55	88.23	87.91	87.82	87.36	87.40	85.63	88.80	88.76	86.90	87.44	89.66	91.21	88.90	88.91	87.60	88.98
1455	372	32	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	86.10	85.05	83.85	86.64	86.52	86.77	84.81	85.53	86.04	85.27	87.08	86.17	84.33	86.16	88.10	85.53	86.00	87.43	87.57	87.25	89.09	88.39	88.83	89.45	87.26
1504	372	32	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	84.84	84.18	90.50	85.84	83.81	87.00	86.81	89.34	86.42	84.02	87.26	88.58	85.80	86.16	85.83	84.44	87.78	86.15	87.88	86.65	86.93	88.91	89.05	82.31	87.56
1507	372	32	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	86.14	85.81	85.80	87.72	86.42	87.76	86.92	86.27	86.83	86.05	86.24	87.43	85.89	84.95	84.77	88.22	86.21	85.13	88.33	86.78	87.22	87.64	88.47	88.21	88.01
1516	372	32	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	83.33	86.61	86.41	85.44	84.88	88.21	85.05	89.00	85.72	84.88	86.28	87.08	87.68	84.68	84.68	87.30	86.22	85.80	87.77	88.61	88.73	88.95	86.24	88.71	88.61
1519	372	32	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	84.64	86.92	85.64	85.74	84.22	85.93	84.97	84.51	84.49	86.25	85.63	88.67	86.93	85.81	85.69	85.89	88.89	85.56	86.56	86.81	87.10	87.86	87.02	88.15	85.87
1568	372	32	0.1	16	0.00001	48	0.000000e+00	0.000000e+00	85.20	85.54	86.16	83.67	84.06	84.38	84.75	86.49	86.59	86.63	86.71	88.15	87.35	85.19	85.03	87.66	83.02	86.77	87.89	89.52	89.24	88.90	87.85	87.89	88.50
1571	372	32	0.1	16	0.00001	48	0.000000e+00	1.000000e-08	83.81	84.43	84.24	83.41	85.02	86.67	84.38	87.81	85.24	83.81	88.73	85.08	86.25	85.41	89.60	85.23	85.50	83.69	88.64	86.82	87.64	89.47	87.09	87.54	88.03

Test Data mAP Results Preview:

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_zero	mAP_test_zero_2	mAP_test_zero_3	mAP_test_zero_4	mAP_test_zero_5	mAP_test_zero_6	mAP_test_zero_7	mAP_test_zero_8	mAP_test_zero_9	mAP_test_zero_10	mAP_test_zero_11	mAP_test_zero_12	mAP_test_zero_13	mAP_test_zero_14	mAP_test_zero_15	mAP_test_zero_16	mAP_test_zero_17	mAP_test_zero_18	mAP_test_zero_19	mAP_test_zero_20	mAP_test_zero_21	mAP_test_zero_22	mAP_test_zero_23	mAP_test_zero_25	mAP_test_zero_26
32	372	32	0.1	16	0.00100	24	0.000000e+00	0.000000e+00	92.61	92.46	92.68	94.10	91.25	90.33	90.41	94.73	92.78	90.30	91.63	91.89	94.07	94.26	91.20	93.94	91.31	94.30	89.63	90.70	91.31	92.61	90.97	92.26	92.21
35	372	32	0.1	16	0.00100	24	0.000000e+00	1.000000e-08	91.71	93.62	93.40	93.21	91.67	92.42	93.39	93.33	94.75	91.95	91.09	91.07	91.88	93.27	94.18	94.42	94.87	48.12	90.92	91.80	91.99	93.83	90.25	90.66	92.82
44	372	32	0.1	16	0.00100	24	1.000000e-08	0.000000e+00	95.13	92.15	94.41	93.79	94.42	93.82	93.92	94.36	90.80	93.44	91.47	92.88	92.38	94.12	92.49	94.18	91.95	93.96	91.34	93.69	92.13	93.64	92.59	91.61	91.95
47	372	32	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	93.75	94.65	94.17	91.66	93.91	78.74	93.57	93.25	93.59	92.58	92.13	91.73	93.34	93.18	92.44	94.31	93.66	91.17	92.38	91.85	92.44	94.99	93.33	93.88	93.20
96	372	32	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	92.01	94.45	95.81	94.50	92.46	90.85	48.12	92.95	95.06	92.90	91.82	92.63	93.17	94.55	93.62	48.12	94.77	94.10	89.93	91.18	90.58	94.28	91.49	91.71	91.86
99	372	32	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	96.24	94.28	93.74	93.54	94.67	91.69	94.19	93.51	94.20	94.43	93.30	92.63	94.54	93.88	94.62	95.29	94.85	95.91	92.85	91.10	91.86	92.90	94.19	92.87	93.95
108	372	32	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	91.97	93.29	95.20	95.17	94.73	93.54	94.81	96.06	93.93	93.73	91.95	91.98	92.66	96.01	94.11	92.69	96.69	94.12	90.08	92.19	91.72	89.79	93.72	92.58	92.09
111	372	32	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	92.90	94.73	95.05	93.16	92.99	92.66	93.63	95.71	94.23	94.66	92.13	94.52	95.37	94.03	94.79	94.91	92.83	94.98	91.58	91.88	92.22	91.94	92.81	89.92	92.10
160	372	32	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	94.59	91.57	94.25	94.74	93.60	92.98	94.56	93.82	94.21	95.10	92.18	93.10	93.09	93.92	95.07	92.03	95.37	92.43	89.68	91.43	91.11	91.70	90.17	91.37	90.81
163	372	32	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	94.39	95.34	94.15	92.02	89.55	93.36	92.39	95.34	93.37	94.20	93.34	92.10	94.56	96.31	93.98	92.95	93.99	93.36	92.53	91.63	84.37	91.51	92.92	94.42	92.25
172	372	32	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	94.23	92.99	94.88	96.27	94.31	93.58	93.28	48.12	95.55	96.52	94.31	94.20	94.38	48.12	95.03	94.22	94.33	92.16	90.20	94.55	92.50	94.96	48.12	93.48	91.57
175	372	32	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	94.40	92.80	93.37	94.31	94.80	92.87	94.21	94.07	94.48	92.80	93.59	90.64	95.40	94.11	92.91	93.96	96.00	94.91	92.48	91.67	91.46	93.80	93.55	92.20	93.57
224	372	32	0.1	16	0.00100	36	0.000000e+00	0.000000e+00	90.71	93.32	91.95	92.73	94.24	91.22	93.97	92.87	92.80	93.01	90.93	91.45	93.43	93.48	90.99	92.17	90.19	92.09	87.87	88.25	93.57	93.65	91.41	91.67	90.63
227	372	32	0.1	16	0.00100	36	0.000000e+00	1.000000e-08	93.45	93.23	94.10	93.84	95.22	92.49	92.14	95.44	94.97	95.43	94.75	93.71	93.80	95.84	92.51	91.43	94.35	93.83	92.55	91.53	94.80	95.42	94.47	92.13	93.40
236	372	32	0.1	16	0.00100	36	1.000000e-08	0.000000e+00	92.99	94.89	91.42	95.47	94.25	89.77	91.13	92.92	93.51	90.38	92.30	89.99	92.33	93.24	93.14	94.01	92.08	93.80	91.89	91.24	92.03	92.31	93.36	91.98	92.06
239	372	32	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	92.84	92.85	94.51	93.86	92.11	91.95	94.81	92.22	94.01	94.10	93.25	92.29	92.61	94.49	94.03	93.40	93.33	92.28	92.25	92.38	91.83	92.48	92.63	92.82	92.05
288	372	32	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	95.06	93.66	92.64	94.35	93.79	91.76	93.33	94.34	93.03	94.49	92.77	91.94	95.74	94.97	93.65	94.27	48.12	93.08	48.12	92.65	92.84	93.17	93.44	93.90	91.70
291	372	32	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	92.44	95.43	93.62	95.52	94.81	90.97	94.45	94.45	94.13	93.79	92.54	93.32	93.86	96.01	95.20	93.30	94.77	95.44	91.05	94.78	94.36	94.26	92.77	90.83	93.97
300	372	32	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	94.85	94.74	95.33	94.14	95.25	91.61	95.01	92.03	93.59	93.02	93.30	93.50	95.51	94.52	95.68	93.92	95.11	93.98	92.03	93.36	94.07	90.19	48.12	92.76	93.73
303	372	32	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	95.33	93.59	94.58	94.56	93.30	92.48	94.29	95.15	93.46	94.34	91.66	91.85	94.60	93.55	95.04	95.06	94.62	95.11	94.10	93.06	93.80	91.94	92.86	93.68	93.18
352	372	32	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	93.26	94.03	93.55	94.20	94.92	93.16	94.20	92.59	93.14	94.83	93.88	93.82	94.01	95.06	94.39	94.96	92.88	94.20	93.04	92.27	92.13	92.31	93.05	92.52	93.97
355	372	32	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	93.75	94.77	94.14	94.83	95.83	92.59	93.83	93.14	94.21	95.06	94.08	93.55	93.84	93.37	93.76	94.80	94.36	95.09	94.61	92.35	91.58	93.54	91.90	92.13	93.05
364	372	32	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	94.50	93.06	93.93	92.05	93.79	91.86	94.61	92.76	94.28	94.54	93.55	93.33	94.35	93.74	91.62	96.44	91.23	94.74	93.42	91.65	92.83	91.70	91.40	92.29	89.95
367	372	32	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	94.87	92.18	94.68	93.76	94.52	91.49	94.84	93.54	94.24	94.48	91.31	93.56	93.74	94.35	93.97	95.81	94.51	94.22	94.28	91.95	93.46	93.75	94.62	94.15	92.30
416	372	32	0.1	16	0.00100	48	0.000000e+00	0.000000e+00	94.62	89.08	93.95	88.54	94.28	92.80	92.65	90.49	94.20	93.77	89.10	94.31	93.81	94.14	89.23	95.09	94.46	93.00	94.61	90.38	92.71	92.31	90.57	92.10	92.63
419	372	32	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	95.72	89.05	92.71	96.39	94.24	90.35	94.08	91.42	94.76	93.86	94.26	91.94	94.04	93.74	93.75	95.57	96.04	94.19	92.54	92.82	92.69	92.36	62.25	94.44	95.04
428	372	32	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	92.26	94.54	94.68	94.41	92.68	93.03	93.09	90.28	93.75	92.50	92.93	90.74	93.65	93.25	92.29	94.24	91.12	94.47	92.84	92.83	95.54	94.09	90.42	92.39	93.96
431	372	32	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	92.90	93.85	94.36	59.29	88.61	93.68	94.08	92.26	95.33	93.85	95.55	92.81	91.96	91.51	95.32	94.30	93.02	94.53	93.82	92.32	91.86	94.39	48.12	94.12	92.80
480	372	32	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	92.66	92.87	95.36	93.35	92.81	91.96	95.18	92.87	95.51	94.32	92.87	93.75	95.71	91.79	94.88	95.65	92.83	94.35	90.32	87.79	93.67	59.06	92.72	85.87	90.89
483	372	32	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	94.28	95.02	94.31	92.50	94.23	92.29	92.47	93.48	48.12	94.08	93.25	90.53	93.82	95.37	93.74	93.98	94.38	94.09	91.01	92.60	57.74	94.20	93.37	92.51	93.20
492	372	32	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	93.93	95.60	95.39	95.52	95.55	93.01	93.62	93.43	93.12	94.12	93.20	93.71	92.63	93.70	95.25	94.24	93.75	90.84	94.29	93.69	93.00	91.90	87.79	93.49	93.08
495	372	32	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	93.54	96.04	92.48	94.26	88.44	94.95	94.31	94.49	93.78	95.03	94.08	93.66	91.24	94.65	92.52	95.01	94.13	93.49	92.78	92.73	92.64	93.41	91.68	93.16	94.19
544	372	32	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	93.22	93.91	95.50	91.66	91.99	93.75	92.59	94.76	94.18	93.32	92.84	94.00	90.82	92.76	92.10	91.50	91.99	93.05	91.44	94.22	91.73	93.61	94.66	93.71	94.89
547	372	32	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	92.16	92.19	93.56	93.63	94.31	91.97	95.74	92.59	92.73	94.42	91.45	93.78	95.29	93.21	92.46	93.78	93.16	94.59	92.95	92.43	92.80	93.70	94.08	92.28	90.86
556	372	32	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	94.48	93.32	91.97	92.43	92.26	92.82	94.14	93.27	95.01	93.09	93.97	92.52	94.53	94.84	92.78	94.64	93.65	95.27	91.49	92.66	91.99	93.50	94.62	94.45	92.81
559	372	32	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	94.03	93.99	92.64	95.00	92.64	90.50	94.46	93.76	94.88	93.08	92.90	94.50	96.10	91.63	92.15	92.79	94.18	94.78	93.79	92.82	93.73	93.68	92.97	91.95	91.58
608	372	32	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	92.59	90.74	91.66	94.35	93.36	92.11	48.12	94.36	90.49	94.08	91.04	91.52	89.85	93.46	92.42	92.39	91.17	92.71	91.56	90.72	92.99	92.98	91.38	90.38	93.51
611	372	32	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	91.08	93.18	94.05	96.16	93.40	92.39	94.74	94.06	95.01	94.87	93.31	89.50	90.65	90.31	92.61	94.54	94.84	92.73	92.73	91.29	94.64	92.66	91.13	93.48	91.69
620	372	32	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	95.76	94.28	94.91	95.31	95.59	92.03	93.64	91.93	95.29	93.58	92.53	93.83	94.28	96.14	94.36	94.78	91.86	95.80	92.94	93.29	93.54	90.06	91.99	92.55	92.52
623	372	32	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	93.92	93.21	94.10	92.59	92.02	94.48	93.55	95.34	91.01	95.31	93.92	92.48	92.92	94.78	93.85	92.84	94.49	92.37	91.25	88.82	92.85	94.42	93.25	91.63	89.24
672	372	32	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	94.48	91.60	93.74	89.44	92.94	91.59	94.35	91.02	93.55	94.52	93.69	92.62	93.53	94.31	93.39	93.47	94.11	93.97	92.24	93.09	91.90	91.85	93.14	91.13	92.91
675	372	32	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	94.03	94.07	94.10	94.43	95.75	91.62	95.15	93.82	94.66	92.47	92.96	93.17	93.57	95.53	92.17	93.85	95.08	93.06	92.89	93.36	90.45	91.23	93.42	91.82	90.62
684	372	32	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	92.38	93.05	88.42	93.66	94.25	92.74	94.46	92.63	94.17	95.72	94.86	94.25	93.02	95.39	94.08	93.09	91.83	91.38	92.77	92.75	92.94	91.54	92.82	91.07	89.67
687	372	32	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	91.48	93.09	93.01	94.00	94.61	90.87	94.85	93.01	90.70	92.92	94.28	93.42	93.58	94.80	91.91	93.43	93.76	92.35	91.77	92.57	92.19	92.42	92.28	93.23	91.90
736	372	32	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	94.03	93.93	92.35	91.22	93.18	93.11	92.04	94.67	93.06	94.29	92.48	93.37	94.26	94.30	91.71	94.44	93.11	93.33	90.71	91.42	91.45	92.09	92.55	93.04	92.79
739	372	32	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	94.06	94.83	95.04	93.21	48.12	93.77	94.52	92.27	92.38	92.60	90.55	91.82	89.68	94.79	94.91	93.81	94.75	94.31	91.43	90.38	94.16	91.74	91.75	91.59	90.22
748	372	32	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	92.74	94.22	93.46	95.22	92.53	92.04	93.58	94.69	92.71	96.04	93.93	92.15	94.49	94.91	95.53	93.52	94.29	94.53	93.26	94.11	91.32	93.87	93.79	92.00	92.74
751	372	32	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	95.93	92.72	93.44	94.55	94.32	93.68	93.50	91.85	93.32	93.44	94.28	91.60	93.42	90.74	93.78	95.19	94.77	91.77	93.52	91.59	91.46	90.60	91.68	91.93	92.41
800	372	32	0.1	16	0.00010	36	0.000000e+00	0.000000e+00	92.23	93.31	91.51	91.90	93.18	89.50	93.63	94.52	93.30	95.54	91.80	90.91	90.31	92.89	93.71	91.59	93.14	91.30	90.75	89.89	89.57	93.61	92.90	94.71	94.64
803	372	32	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	94.52	95.39	93.52	95.80	93.34	91.79	93.32	94.89	93.39	92.39	91.62	92.96	92.03	94.35	91.54	93.31	94.28	93.18	90.83	86.72	92.39	91.90	92.13	93.95	94.04
812	372	32	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	93.49	91.90	94.34	93.53	91.76	92.52	93.64	95.59	93.42	95.26	90.91	92.18	91.72	93.81	94.65	93.94	96.73	94.04	92.76	93.91	93.59	93.61	93.55	93.90	94.09
815	372	32	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	90.65	91.62	93.12	93.04	92.51	91.07	91.79	93.87	91.26	92.40	90.82	91.01	92.63	92.94	92.10	92.54	93.06	90.74	91.10	87.13	92.18	92.14	91.81	93.43	89.28
864	372	32	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	93.83	94.03	93.52	94.27	93.35	92.66	93.31	92.69	48.12	93.81	92.47	93.11	93.16	94.47	92.19	95.49	94.22	94.57	91.50	89.61	90.99	91.07	92.32	91.85	91.89
867	372	32	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	92.63	94.45	93.06	95.01	92.78	91.66	95.53	95.57	93.90	94.03	90.81	92.91	94.30	93.96	92.18	92.93	93.81	93.96	92.18	93.70	93.41	90.71	92.59	91.78	91.79
876	372	32	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	94.33	91.61	94.66	94.61	95.07	93.82	93.89	95.16	93.19	94.67	94.23	92.95	93.75	93.73	91.78	93.70	95.56	95.54	91.51	93.13	90.53	93.09	91.25	92.73	94.47
879	372	32	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	94.52	94.16	95.87	93.59	95.56	93.35	92.64	93.14	93.82	94.03	91.79	94.10	95.11	93.91	95.33	95.54	95.42	95.09	91.47	91.45	92.30	95.66	92.21	93.56	92.32
928	372	32	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	93.60	92.88	94.32	93.58	93.80	92.01	96.49	93.76	95.48	95.24	93.52	93.94	93.89	94.76	93.15	94.67	96.13	95.81	91.59	92.72	92.88	91.63	92.28	91.97	94.88
931	372	32	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	94.12	94.91	95.40	93.93	95.79	93.92	93.97	94.39	93.83	93.59	93.01	92.42	93.93	93.29	92.51	94.42	95.00	93.46	92.89	90.39	92.88	92.91	92.55	91.73	91.38
940	372	32	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	94.61	94.72	93.77	95.91	95.00	93.76	93.65	94.02	94.28	94.38	93.28	93.54	95.53	93.56	93.77	93.46	93.68	95.38	92.95	93.97	92.16	91.97	93.97	94.39	90.02
943	372	32	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	94.05	94.74	94.26	95.99	95.46	92.21	91.99	92.15	94.90	94.63	93.71	94.31	93.58	94.11	92.82	93.80	94.96	94.75	94.10	93.62	94.03	92.77	92.59	92.25	93.59
992	372	32	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	95.67	92.26	59.72	84.95	65.55	92.47	92.88	93.05	93.19	94.95	94.90	93.10	92.15	95.00	92.03	94.02	89.72	93.96	92.66	92.59	94.04	93.26	92.33	93.58	84.85
995	372	32	0.1	16	0.00010	48	0.000000e+00	1.000000e-08	92.79	89.22	89.69	95.55	93.00	93.47	94.58	94.55	92.73	92.76	92.31	92.51	95.30	91.85	94.91	93.89	92.51	95.97	92.17	93.26	92.01	93.52	91.82	92.31	94.33
1004	372	32	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	91.86	95.29	84.18	94.45	92.94	94.52	94.70	94.07	95.07	94.59	91.68	90.98	85.84	95.43	79.41	94.98	95.00	95.47	93.42	94.24	94.58	94.80	92.39	92.88	87.90
1007	372	32	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	94.82	93.60	94.49	95.45	91.92	94.11	93.50	93.82	92.77	94.55	92.20	94.62	93.91	92.74	92.38	92.97	93.23	92.67	93.46	93.53	93.87	92.72	89.51	93.54	92.53
1056	372	32	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	94.49	58.74	94.03	94.77	91.86	90.91	95.34	92.94	94.22	93.72	92.89	92.00	92.31	93.54	93.14	93.06	94.18	91.88	93.47	93.51	61.54	93.20	94.62	90.74	91.61
1059	372	32	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	93.50	94.48	95.72	94.29	94.50	91.88	93.05	95.51	94.71	93.83	92.54	93.35	95.01	91.01	93.83	94.80	92.96	94.11	93.33	90.77	93.56	91.70	92.99	85.10	91.60
1068	372	32	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	94.53	91.72	94.30	92.52	95.20	92.52	95.13	94.01	93.54	94.36	91.32	93.27	95.72	92.35	90.60	95.61	94.72	94.00	94.06	92.88	92.84	93.28	94.04	93.79	93.02
1071	372	32	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	95.49	93.46	95.25	95.08	94.41	92.63	93.14	94.79	93.76	94.43	93.12	93.58	94.06	92.81	93.52	94.17	93.53	94.77	93.31	92.63	94.76	93.19	59.16	91.71	92.85
1120	372	32	0.1	64	0.00010	48	0.000000e+00	0.000000e+00	93.38	92.09	94.47	94.68	92.73	92.41	94.57	93.56	94.73	95.25	92.29	93.20	95.03	95.60	93.29	92.99	92.08	90.65	93.31	90.31	87.66	94.61	94.05	92.37	91.06
1123	372	32	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	93.94	95.56	94.63	95.82	93.28	93.09	94.37	94.21	90.55	94.30	92.60	92.95	94.24	94.32	92.88	92.06	93.49	93.30	92.21	91.82	92.93	94.05	94.45	91.26	93.50
1132	372	32	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	93.87	93.05	93.47	92.53	93.71	91.57	93.25	93.79	94.58	94.08	93.85	92.04	93.97	92.73	92.58	94.04	93.84	92.58	91.50	93.09	95.14	94.08	94.81	94.49	93.47
1135	372	32	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	92.27	95.17	93.79	93.81	94.48	91.81	94.70	92.62	95.45	92.59	93.04	94.49	93.38	93.11	91.70	95.94	93.29	93.75	92.62	93.04	93.96	94.52	93.33	91.51	90.11
1184	372	32	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	96.00	94.11	90.79	90.99	93.81	91.26	90.64	92.37	92.53	91.43	91.64	92.02	92.74	93.92	90.88	88.60	94.05	92.35	91.83	90.69	93.56	91.23	91.22	89.18	89.42
1187	372	32	0.1	16	0.00001	24	0.000000e+00	1.000000e-08	94.64	93.71	92.24	92.90	96.68	93.87	93.10	93.66	94.24	92.12	90.48	90.54	95.56	92.01	91.89	92.30	94.93	92.83	92.77	91.03	91.48	94.22	92.97	92.65	89.55
1196	372	32	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	93.45	95.36	94.42	93.48	92.82	92.73	94.31	94.72	93.37	94.00	94.23	90.70	93.57	95.08	93.68	91.72	93.20	93.17	93.52	92.05	91.95	95.35	94.11	92.81	94.36
1199	372	32	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	91.76	95.23	91.76	95.32	93.03	91.34	94.61	95.55	94.88	93.19	91.45	92.70	90.99	93.32	94.02	95.00	84.26	94.07	92.76	93.71	93.09	93.64	93.40	90.05	93.13
1248	372	32	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	93.13	94.21	93.71	93.62	93.00	90.07	92.64	93.23	94.11	95.13	92.42	94.14	94.27	94.89	91.34	93.74	93.77	94.47	90.42	91.19	93.26	91.37	92.63	92.17	91.84
1251	372	32	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	94.44	92.24	94.31	94.50	93.83	93.81	94.41	92.04	94.51	93.42	92.52	91.59	92.63	95.50	94.72	91.14	94.57	94.16	91.56	90.30	92.28	91.72	92.78	92.49	91.60
1260	372	32	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	92.88	94.13	95.53	90.67	93.60	92.63	93.09	93.62	93.55	93.08	94.77	93.14	90.64	94.24	94.84	94.68	94.79	95.50	89.87	93.55	92.24	92.13	92.46	90.72	93.48
1263	372	32	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	92.24	91.36	93.46	92.89	94.91	90.76	93.33	91.58	94.13	94.35	94.40	90.79	94.05	94.40	93.47	89.56	95.03	92.40	91.81	91.88	92.44	93.02	92.71	91.01	91.50
1312	372	32	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	92.69	93.78	92.78	93.31	91.01	92.13	93.40	91.87	93.85	93.45	92.19	92.92	93.42	95.66	91.26	94.35	94.35	92.20	92.64	91.42	92.33	90.51	92.75	92.37	92.70
1315	372	32	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	92.99	94.02	95.40	94.12	95.78	92.23	92.53	94.95	93.80	94.45	92.82	93.72	90.93	95.02	93.86	95.74	95.97	95.05	94.23	92.61	93.05	91.20	92.27	91.57	93.67
1324	372	32	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	94.26	92.24	94.35	94.65	91.70	93.59	92.43	95.59	94.99	93.80	91.72	93.31	96.14	94.06	92.68	94.39	94.28	92.90	92.92	91.54	92.77	94.28	93.58	91.70	92.11
1327	372	32	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	95.61	95.25	93.63	91.83	94.76	93.14	94.43	95.47	91.66	95.38	93.19	92.78	93.48	92.43	93.61	92.66	95.82	94.15	91.56	92.42	93.36	93.63	93.23	91.80	89.53
1376	372	32	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	92.86	94.31	91.89	95.37	93.91	92.25	92.94	93.19	93.43	96.00	91.70	91.75	95.09	92.76	92.94	91.08	91.83	93.81	89.01	89.94	93.65	93.18	94.92	93.94	92.19
1379	372	32	0.1	16	0.00001	36	0.000000e+00	1.000000e-08	94.06	95.71	93.56	96.68	93.83	91.54	94.55	95.15	93.86	94.22	93.30	80.96	93.91	94.15	93.46	93.90	95.84	93.35	92.46	90.71	94.49	93.49	90.77	94.97	91.92
1388	372	32	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	92.33	93.32	93.46	95.36	93.09	93.24	92.52	91.54	94.67	93.67	93.24	93.15	94.01	93.39	95.40	94.16	94.45	90.74	92.93	92.55	94.74	89.67	93.02	91.13	93.53
1391	372	32	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	93.84	92.62	92.51	95.54	93.24	94.25	92.34	93.71	93.40	95.17	92.16	92.96	93.66	92.87	95.03	94.01	94.99	93.83	93.26	91.52	95.06	93.14	92.44	93.42	93.09
1440	372	32	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	93.12	93.72	93.16	94.74	85.22	92.12	94.60	92.23	95.45	94.49	92.12	91.59	93.46	95.04	93.66	93.89	94.47	90.54	92.70	91.79	92.80	92.96	93.11	91.20	93.05
1443	372	32	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	93.92	94.78	92.99	94.13	94.45	92.31	93.41	94.51	94.81	93.46	93.65	93.22	94.46	94.40	91.74	94.15	95.55	94.45	93.40	92.13	92.65	90.48	91.77	90.04	91.65
1452	372	32	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	93.27	94.93	94.69	95.47	94.97	92.47	95.77	95.05	95.43	95.00	91.42	93.87	93.71	94.40	93.23	94.56	95.02	94.28	93.28	91.94	91.90	91.85	92.83	92.95	91.62
1455	372	32	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	95.61	95.98	93.77	95.57	94.81	92.72	95.80	95.10	95.10	94.76	95.15	93.57	94.05	94.41	92.36	92.90	94.83	92.49	94.75	91.18	92.79	93.13	92.19	94.19	93.50
1504	372	32	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	93.84	95.55	95.06	94.12	92.32	93.82	93.79	93.99	95.57	94.44	92.95	95.60	95.64	94.16	95.03	94.99	95.18	96.05	92.68	93.14	93.44	93.37	92.30	94.64	93.46
1507	372	32	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	92.69	95.58	96.13	95.72	94.25	92.96	95.15	93.44	94.35	93.70	94.06	93.83	93.76	94.16	93.66	94.49	96.80	93.53	93.94	92.69	93.05	93.72	93.34	91.55	90.87
1516	372	32	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	94.23	94.15	96.01	94.96	95.46	92.45	94.39	93.12	95.32	93.90	94.10	93.46	93.68	93.84	94.45	93.76	95.54	95.13	93.59	93.49	92.23	92.55	93.41	92.00	94.18
1519	372	32	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	92.27	92.65	94.82	92.46	96.34	92.19	94.43	95.78	91.66	92.88	92.41	93.51	93.00	94.65	92.76	94.50	95.56	93.36	92.73	92.40	93.01	93.50	92.15	89.64	92.78
1568	372	32	0.1	16	0.00001	48	0.000000e+00	0.000000e+00	95.15	71.94	93.46	94.14	90.25	93.09	92.57	94.05	89.97	94.60	92.39	92.36	93.55	88.88	95.59	90.92	94.67	91.46	92.23	94.46	92.83	92.94	93.56	92.60	62.34
1571	372	32	0.1	16	0.00001	48	0.000000e+00	1.000000e-08	93.64	94.70	93.58	93.16	94.85	93.75	94.39	94.35	95.48	92.47	93.03	93.49	95.80	92.90	93.78	92.34	94.52	93.83	91.67	91.11	93.48	92.29	93.47	90.17	91.98

Model performance on validation data (T-test)

Statistical Significance:

Using the global mean maximum row as the reference, we perform the right-tailed t-test to identify significant hyperparameters.

pvalues = []
pvalues_real = []

for _,index in enumerate(dt_valid_sub.index):
  _, p_value = stats.ttest_ind(max_row[:-1], dt_valid_sub.loc[index][:-1], alternative='greater')
  pvalues_real.append(p_value)
  pvalues.append((p_value >= 0.05)*1)

  

dt_valid_sub['pvalues'] = pvalues_real
dt_valid_sub['sig'] = pvalues


dt_valid_sub1 = dt_valid_sub.query('sig == 1')


max_indexx = dt_valid_sub1.iloc[:, :-3].mean().idxmax()

#print(sum(pvalues),max_indexx)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_zero	mAP_valid_zero_2	mAP_valid_zero_3	mAP_valid_zero_4	mAP_valid_zero_5	mAP_valid_zero_6	mAP_valid_zero_7	mAP_valid_zero_8	mAP_valid_zero_9	mAP_valid_zero_10	mAP_valid_zero_11	mAP_valid_zero_12	mAP_valid_zero_13	mAP_valid_zero_14	mAP_valid_zero_15	mAP_valid_zero_16	mAP_valid_zero_17	mAP_valid_zero_18	mAP_valid_zero_19	mAP_valid_zero_20	mAP_valid_zero_21	mAP_valid_zero_22	mAP_valid_zero_23	mAP_valid_zero_25	mAP_valid_zero_26
431	372	32	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	85.72	85.64	86.08	86.95	84.17	87.46	84.13	85.69	85.26	87.48	88.10	89.88	87.50	85.01	85.96	86.58	90.29	85.16	89.55	87.85	90.40	88.13	43.72	87.04	88.91
687	372	32	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	86.16	86.66	87.05	86.72	86.68	86.93	86.20	86.77	88.17	86.60	88.53	89.26	86.85	86.31	87.84	87.96	86.16	87.39	89.55	88.46	87.63	89.79	90.02	88.27	88.81
1059	372	32	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	86.63	85.26	87.13	84.91	86.23	87.25	87.50	86.35	86.16	87.70	88.94	88.08	86.48	86.69	86.20	88.35	87.90	87.53	90.55	88.22	87.47	89.38	89.85	89.09	89.53
1068	372	32	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	87.26	87.64	86.94	85.23	87.37	86.72	86.13	87.41	88.48	87.73	88.42	88.20	89.48	85.12	87.28	88.36	86.92	87.45	89.00	89.17	88.41	89.15	88.13	89.70	90.25
1071	372	32	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	85.91	85.36	88.31	87.63	89.53	86.65	87.26	86.69	87.87	87.72	87.56	86.75	87.83	86.44	86.72	86.26	86.41	87.73	87.01	88.64	89.37	88.74	90.47	89.69	87.35
111	372	32	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	84.30	84.87	87.86	86.25	88.54	87.57	87.32	85.68	89.37	88.30	88.20	89.58	89.44	86.68	87.06	86.35	86.01	88.71	87.18	87.60	90.85	85.89	88.69	88.60	89.95
480	372	32	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	86.17	84.98	86.86	86.45	86.11	86.21	84.72	86.26	86.57	87.74	89.45	88.88	86.63	85.74	86.97	87.84	85.60	88.13	89.07	89.96	87.34	92.58	88.91	88.19	89.47
483	372	32	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	85.38	86.74	87.22	87.06	87.69	89.67	83.53	86.28	43.72	87.40	89.52	89.77	86.61	87.14	87.93	87.40	85.41	87.50	90.02	87.86	90.82	88.34	87.78	86.14	89.74
495	372	32	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	85.36	85.76	85.16	86.10	83.23	88.14	89.99	87.93	87.86	88.02	86.47	89.21	90.60	86.89	89.18	87.73	87.43	87.74	88.73	91.18	89.22	88.78	90.97	87.81	89.84
1327	372	32	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	85.33	86.59	86.04	86.11	85.20	87.93	86.65	87.54	85.82	90.05	88.39	87.46	88.11	87.05	85.99	87.61	88.35	86.87	87.54	88.55	89.82	89.05	88.02	86.67	91.92
739	372	32	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	86.08	86.03	85.85	88.05	43.72	87.04	86.76	86.55	87.97	87.70	87.19	87.72	86.59	87.14	85.52	86.54	84.93	88.22	85.70	88.45	89.70	88.59	88.98	86.52	89.49
1123	372	32	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	88.22	85.28	87.40	86.97	85.51	90.35	86.11	88.33	86.26	86.83	87.45	88.38	88.58	87.14	88.45	86.83	86.75	88.04	88.04	88.06	89.53	90.14	90.03	89.96	89.31
1135	372	32	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	86.14	86.91	88.76	84.75	88.43	87.26	84.35	86.14	85.73	87.43	87.71	87.63	87.28	87.85	88.83	87.53	86.64	86.60	89.99	91.21	88.72	90.77	87.78	91.36	90.53
547	372	32	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	86.14	87.66	86.50	87.01	87.47	89.74	87.98	87.14	87.17	89.51	88.39	89.23	88.88	87.79	86.27	90.24	85.51	84.89	92.18	89.88	88.57	88.38	89.94	91.49	90.34

Size of the All data:  (98, 28)
Size of the Sig data:  (14, 28)

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params1 = dt_valid.loc[dt_valid_sub1.index][['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
optimal_params = optimal_params1.reset_index()
optimal_params.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_1 = pd.merge(optimal_params, dt_test, on=['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg'])
test_data_1_sub = test_data_1[list(test_data_1.columns[test_data_1.columns.str.startswith('mAP_test')])]

Equivalent test data

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_zero	mAP_test_zero_2	mAP_test_zero_3	mAP_test_zero_4	mAP_test_zero_5	mAP_test_zero_6	mAP_test_zero_7	mAP_test_zero_8	mAP_test_zero_9	mAP_test_zero_10	mAP_test_zero_11	mAP_test_zero_12	mAP_test_zero_13	mAP_test_zero_14	mAP_test_zero_15	mAP_test_zero_16	mAP_test_zero_17	mAP_test_zero_18	mAP_test_zero_19	mAP_test_zero_20	mAP_test_zero_21	mAP_test_zero_22	mAP_test_zero_23	mAP_test_zero_25	mAP_test_zero_26
13	372	32	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	92.90	93.85	94.36	59.29	88.61	93.68	94.08	92.26	95.33	93.85	95.55	92.81	91.96	91.51	95.32	94.30	93.02	94.53	93.82	92.32	91.86	94.39	48.12	94.12	92.80
6	372	32	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	91.48	93.09	93.01	94.00	94.61	90.87	94.85	93.01	90.70	92.92	94.28	93.42	93.58	94.80	91.91	93.43	93.76	92.35	91.77	92.57	92.19	92.42	92.28	93.23	91.90
8	372	32	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	93.50	94.48	95.72	94.29	94.50	91.88	93.05	95.51	94.71	93.83	92.54	93.35	95.01	91.01	93.83	94.80	92.96	94.11	93.33	90.77	93.56	91.70	92.99	85.10	91.60
4	372	32	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	94.53	91.72	94.30	92.52	95.20	92.52	95.13	94.01	93.54	94.36	91.32	93.27	95.72	92.35	90.60	95.61	94.72	94.00	94.06	92.88	92.84	93.28	94.04	93.79	93.02
7	372	32	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	95.49	93.46	95.25	95.08	94.41	92.63	93.14	94.79	93.76	94.43	93.12	93.58	94.06	92.81	93.52	94.17	93.53	94.77	93.31	92.63	94.76	93.19	59.16	91.71	92.85
5	372	32	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	92.90	94.73	95.05	93.16	92.99	92.66	93.63	95.71	94.23	94.66	92.13	94.52	95.37	94.03	94.79	94.91	92.83	94.98	91.58	91.88	92.22	91.94	92.81	89.92	92.10
10	372	32	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	92.66	92.87	95.36	93.35	92.81	91.96	95.18	92.87	95.51	94.32	92.87	93.75	95.71	91.79	94.88	95.65	92.83	94.35	90.32	87.79	93.67	59.06	92.72	85.87	90.89
11	372	32	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	94.28	95.02	94.31	92.50	94.23	92.29	92.47	93.48	48.12	94.08	93.25	90.53	93.82	95.37	93.74	93.98	94.38	94.09	91.01	92.60	57.74	94.20	93.37	92.51	93.20
1	372	32	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	93.54	96.04	92.48	94.26	88.44	94.95	94.31	94.49	93.78	95.03	94.08	93.66	91.24	94.65	92.52	95.01	94.13	93.49	92.78	92.73	92.64	93.41	91.68	93.16	94.19
9	372	32	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	95.61	95.25	93.63	91.83	94.76	93.14	94.43	95.47	91.66	95.38	93.19	92.78	93.48	92.43	93.61	92.66	95.82	94.15	91.56	92.42	93.36	93.63	93.23	91.80	89.53
12	372	32	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	94.06	94.83	95.04	93.21	48.12	93.77	94.52	92.27	92.38	92.60	90.55	91.82	89.68	94.79	94.91	93.81	94.75	94.31	91.43	90.38	94.16	91.74	91.75	91.59	90.22
2	372	32	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	93.94	95.56	94.63	95.82	93.28	93.09	94.37	94.21	90.55	94.30	92.60	92.95	94.24	94.32	92.88	92.06	93.49	93.30	92.21	91.82	92.93	94.05	94.45	91.26	93.50
3	372	32	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	92.27	95.17	93.79	93.81	94.48	91.81	94.70	92.62	95.45	92.59	93.04	94.49	93.38	93.11	91.70	95.94	93.29	93.75	92.62	93.04	93.96	94.52	93.33	91.51	90.11
0	372	32	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	92.16	92.19	93.56	93.63	94.31	91.97	95.74	92.59	92.73	94.42	91.45	93.78	95.29	93.21	92.46	93.78	93.16	94.59	92.95	92.43	92.80	93.70	94.08	92.28	90.86

Size of the test data:  (14, 33)

Difference

diff = dt[['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
for i, cols in enumerate(zip(dt.columns,test_data_1.columns)):   
   diff[f'col_diff_{i}'] = (np.array(dt[cols[0]]) - np.array(test_data_1[cols[1]]))*-1

# Function to count negatives in a row
def count_negatives(row):
    return sum(1 for value in row if isinstance(value, (int, float)) and value < 0)

diff['negative_count'] = diff.apply(count_negatives, axis=1)
diff.sort_values(by =["negative_count"],inplace = True, ascending=False)

html_table = diff.to_html(index=True)

# Wrap in a scrollable div
scrollable_table = f"""
<div style="height: 400px; width: 100%; overflow-x: auto; overflow-y: auto;">
    {html_table}
</div>
"""
# Display the scrollable table
display(HTML(scrollable_table))

# 
# show(diff)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	col_diff_0	col_diff_1	col_diff_2	col_diff_3	col_diff_4	col_diff_5	col_diff_6	col_diff_7	col_diff_8	col_diff_9	col_diff_10	col_diff_11	col_diff_12	col_diff_13	col_diff_14	col_diff_15	col_diff_16	col_diff_17	col_diff_18	col_diff_19	col_diff_20	col_diff_21	col_diff_22	col_diff_23	col_diff_24	col_diff_25	col_diff_26	col_diff_27	col_diff_28	col_diff_29	col_diff_30	col_diff_31	col_diff_32	negative_count
480	372	32	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.49	7.89	8.50	6.90	6.70	5.75	10.46	6.61	8.94	6.58	3.42	4.87	9.08	6.05	7.91	7.81	7.23	6.22	1.25	-2.17	6.33	-33.52	3.81	-2.32	1.42	3
431	372	32	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.18	8.21	8.28	-27.66	4.44	6.22	9.95	6.57	10.07	6.37	7.45	2.93	4.46	6.50	9.36	7.72	2.73	9.37	4.27	4.47	1.46	6.26	4.40	7.08	3.89	1
1059	372	32	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.87	9.22	8.59	9.38	8.27	4.63	5.55	9.16	8.55	6.13	3.60	5.27	8.53	4.32	7.63	6.45	5.06	6.58	2.78	2.55	6.09	2.32	3.14	-3.99	2.07	1
1071	372	32	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.58	8.10	6.94	7.45	4.88	5.98	5.88	8.10	5.89	6.71	5.56	6.83	6.23	6.37	6.80	7.91	7.12	7.04	6.30	3.99	5.39	4.45	-31.31	2.02	5.50	1
483	372	32	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.90	8.28	7.09	5.44	6.54	2.62	8.94	7.20	4.40	6.68	3.73	0.76	7.21	8.23	5.81	6.58	8.97	6.59	0.99	4.74	-33.08	5.86	5.59	6.37	3.46	1
1327	372	32	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	10.28	8.66	7.59	5.72	9.56	5.21	7.78	7.93	5.84	5.33	4.80	5.32	5.37	5.38	7.62	5.05	7.47	7.28	4.02	3.87	3.54	4.58	5.21	5.13	-2.39	1
1135	372	32	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.13	8.26	5.03	9.06	6.05	4.55	10.35	6.48	9.72	5.16	5.33	6.86	6.10	5.26	2.87	8.41	6.65	7.15	2.63	1.83	5.24	3.75	5.55	0.15	-0.42	1
687	372	32	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.32	6.43	5.96	7.28	7.93	3.94	8.65	6.24	2.53	6.32	5.75	4.16	6.73	8.49	4.07	5.47	7.60	4.96	2.22	4.11	4.56	2.63	2.26	4.96	3.09	0
1068	372	32	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.27	4.08	7.36	7.29	7.83	5.80	9.00	6.60	5.06	6.63	2.90	5.07	6.24	7.23	3.32	7.25	7.80	6.55	5.06	3.71	4.43	4.13	5.91	4.09	2.77	0
111	372	32	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.60	9.86	7.19	6.91	4.45	5.09	6.31	10.03	4.86	6.36	3.93	4.94	5.93	7.35	7.73	8.56	6.82	6.27	4.40	4.28	1.37	6.05	4.12	1.32	2.15	0
495	372	32	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.18	10.28	7.32	8.16	5.21	6.81	4.32	6.56	5.92	7.01	7.61	4.45	0.64	7.76	3.34	7.28	6.70	5.75	4.05	1.55	3.42	4.63	0.71	5.35	4.35	0
739	372	32	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.98	8.80	9.19	5.16	4.40	6.73	7.76	5.72	4.41	4.90	3.36	4.10	3.09	7.65	9.39	7.27	9.82	6.09	5.73	1.93	4.46	3.15	2.77	5.07	0.73	0
1123	372	32	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.72	10.28	7.23	8.85	7.77	2.74	8.26	5.88	4.29	7.47	5.15	4.57	5.66	7.18	4.43	5.23	6.74	5.26	4.17	3.76	3.40	3.91	4.42	1.30	4.19	0
547	372	32	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.02	4.53	7.06	6.62	6.84	2.23	7.76	5.45	5.56	4.91	3.06	4.55	6.41	5.42	6.19	3.54	7.65	9.70	0.77	2.55	4.23	5.32	4.14	0.79	0.52	0

Model performance on valid data

Average & Std Deviation of the Significant rows:

	mean	std
mAP_valid_zero_26	89.674286	1.023589
mAP_valid_zero_21	89.132143	1.165657
mAP_valid_zero_22	89.122143	1.502988
mAP_valid_zero_20	88.935000	1.186545
mAP_valid_zero_19	88.865000	1.668227
mAP_valid_zero_25	88.609286	1.707312
mAP_valid_zero_12	88.573571	0.967038
mAP_valid_zero_11	88.165714	0.845838
mAP_valid_zero_13	87.918571	1.309180
mAP_valid_zero_10	87.872143	0.921004
mAP_valid_zero_6	87.780000	1.271099
mAP_valid_zero_16	87.541429	1.053411
mAP_valid_zero_18	87.282857	1.096550
mAP_valid_zero_15	87.157143	1.145388
mAP_valid_zero_3	86.940000	0.979796
mAP_valid_zero_8	86.768571	0.803903
mAP_valid_zero_17	86.736429	1.411337
mAP_valid_zero_14	86.642143	0.863688
mAP_valid_zero_4	86.442143	0.971067
mAP_valid_zero_7	86.330714	1.730209
mAP_valid_zero_2	86.098571	0.938475
mAP_valid_zero	86.057143	0.927639
mAP_valid_zero_23	85.949286	12.198771
mAP_valid_zero_9	84.029286	11.664278
mAP_valid_zero_5	83.562857	11.597691

Model performance on Test data

We then apply the hyperparameters to the test set and average the results.

test_data_1['average_map'] = test_data_1_sub.apply(np.mean, axis=1)

test_data_1.sort_values(by=['average_map'], ascending= False,inplace=True)

	mean	std
mAP_test_zero_3	94.320714	0.940225
mAP_test_zero_16	94.293571	1.124815
mAP_test_zero_7	94.257143	0.911393
mAP_test_zero_2	94.161429	1.316989
mAP_test_zero_18	94.055000	0.675753
mAP_test_zero_10	94.055000	0.841891
mAP_test_zero_8	93.806429	1.246204
mAP_test_zero_17	93.762143	0.889712
mAP_test_zero_13	93.752857	1.774930
mAP_test_zero	93.522857	1.221521
mAP_test_zero_15	93.333571	1.394073
mAP_test_zero_14	93.298571	1.381826
mAP_test_zero_12	93.193571	1.034070
mAP_test_zero_11	92.855000	1.286030
mAP_test_zero_6	92.658571	1.022454
average_map	92.520229	1.243488
mAP_test_zero_19	92.339286	1.106592
mAP_test_zero_26	91.912143	1.423269
mAP_test_zero_20	91.875714	1.404015
mAP_test_zero_25	91.275000	2.690867
mAP_test_zero_4	91.196429	9.242248
mAP_test_zero_22	90.802143	9.186058
mAP_test_zero_21	90.620714	9.499128
mAP_test_zero_9	90.175000	12.214988
mAP_test_zero_5	90.053571	12.256997
mAP_test_zero_23	87.429286	14.500629

Model performance on validation data (Mann-Whitney U Test)

Statistical Significance:

Using the global mean maximum row as the reference, we perform the right-tailed t-test to identify significant hyperparameters.

pvalues_mw = []
pvalues_real_mw = []

for _,index in enumerate(dt_valid_sub.index):

  _, p_value_mw = stats.mannwhitneyu(max_row[:-1], dt_valid_sub.loc[index][:-1], alternative='greater')#, method = 'asymptotic')
  pvalues_real_mw.append(p_value_mw)
  pvalues_mw.append((p_value_mw >= 0.05)*1)
  

dt_valid_sub_mw['pvalues_mw'] = pvalues_real_mw
dt_valid_sub_mw['sig_mw'] = pvalues_mw

dt_valid_sub_mw1 = dt_valid_sub_mw.query('sig_mw == 1')


max_indexx = dt_valid_sub_mw1.iloc[:, :-3].mean().idxmax()

#print(sum(pvalues),max_indexx)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_zero	mAP_valid_zero_2	mAP_valid_zero_3	mAP_valid_zero_4	mAP_valid_zero_5	mAP_valid_zero_6	mAP_valid_zero_7	mAP_valid_zero_8	mAP_valid_zero_9	mAP_valid_zero_10	mAP_valid_zero_11	mAP_valid_zero_12	mAP_valid_zero_13	mAP_valid_zero_14	mAP_valid_zero_15	mAP_valid_zero_16	mAP_valid_zero_17	mAP_valid_zero_18	mAP_valid_zero_19	mAP_valid_zero_20	mAP_valid_zero_21	mAP_valid_zero_22	mAP_valid_zero_23	mAP_valid_zero_25	mAP_valid_zero_26
1068	372	32	0.1	32	0.0001	48	1.000000e-08	0.000000e+00	87.26	87.64	86.94	85.23	87.37	86.72	86.13	87.41	88.48	87.73	88.42	88.20	89.48	85.12	87.28	88.36	86.92	87.45	89.00	89.17	88.41	89.15	88.13	89.70	90.25
111	372	32	0.1	32	0.0010	24	1.000000e-08	1.000000e-08	84.30	84.87	87.86	86.25	88.54	87.57	87.32	85.68	89.37	88.30	88.20	89.58	89.44	86.68	87.06	86.35	86.01	88.71	87.18	87.60	90.85	85.89	88.69	88.60	89.95
495	372	32	0.1	32	0.0010	48	1.000000e-08	1.000000e-08	85.36	85.76	85.16	86.10	83.23	88.14	89.99	87.93	87.86	88.02	86.47	89.21	90.60	86.89	89.18	87.73	87.43	87.74	88.73	91.18	89.22	88.78	90.97	87.81	89.84
1123	372	32	0.1	64	0.0001	48	0.000000e+00	1.000000e-08	88.22	85.28	87.40	86.97	85.51	90.35	86.11	88.33	86.26	86.83	87.45	88.38	88.58	87.14	88.45	86.83	86.75	88.04	88.04	88.06	89.53	90.14	90.03	89.96	89.31
1135	372	32	0.1	64	0.0001	48	1.000000e-08	1.000000e-08	86.14	86.91	88.76	84.75	88.43	87.26	84.35	86.14	85.73	87.43	87.71	87.63	87.28	87.85	88.83	87.53	86.64	86.60	89.99	91.21	88.72	90.77	87.78	91.36	90.53
547	372	32	0.1	64	0.0010	48	0.000000e+00	1.000000e-08	86.14	87.66	86.50	87.01	87.47	89.74	87.98	87.14	87.17	89.51	88.39	89.23	88.88	87.79	86.27	90.24	85.51	84.89	92.18	89.88	88.57	88.38	89.94	91.49	90.34

Size of the All data:  (98, 28)
Size of the Sig data:  (6, 33)

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params_mw1 = dt_valid.loc[dt_valid_sub_mw1.index][['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
optimal_params_mw = optimal_params_mw1.reset_index()
optimal_params_mw.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_mw1 = pd.merge(optimal_params_mw, dt_test, on=['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg'])
test_data_1_sub_mw = test_data_mw1[list(test_data_mw1.columns[test_data_mw1.columns.str.startswith('mAP_test')])]

Equivalent test data

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_zero	mAP_test_zero_2	mAP_test_zero_3	mAP_test_zero_4	mAP_test_zero_5	mAP_test_zero_6	mAP_test_zero_7	mAP_test_zero_8	mAP_test_zero_9	mAP_test_zero_10	mAP_test_zero_11	mAP_test_zero_12	mAP_test_zero_13	mAP_test_zero_14	mAP_test_zero_15	mAP_test_zero_16	mAP_test_zero_17	mAP_test_zero_18	mAP_test_zero_19	mAP_test_zero_20	mAP_test_zero_21	mAP_test_zero_22	mAP_test_zero_23	mAP_test_zero_25	mAP_test_zero_26
4	372	32	0.1	32	0.0001	48	1.000000e-08	0.000000e+00	94.53	91.72	94.30	92.52	95.20	92.52	95.13	94.01	93.54	94.36	91.32	93.27	95.72	92.35	90.60	95.61	94.72	94.00	94.06	92.88	92.84	93.28	94.04	93.79	93.02
5	372	32	0.1	32	0.0010	24	1.000000e-08	1.000000e-08	92.90	94.73	95.05	93.16	92.99	92.66	93.63	95.71	94.23	94.66	92.13	94.52	95.37	94.03	94.79	94.91	92.83	94.98	91.58	91.88	92.22	91.94	92.81	89.92	92.10
1	372	32	0.1	32	0.0010	48	1.000000e-08	1.000000e-08	93.54	96.04	92.48	94.26	88.44	94.95	94.31	94.49	93.78	95.03	94.08	93.66	91.24	94.65	92.52	95.01	94.13	93.49	92.78	92.73	92.64	93.41	91.68	93.16	94.19
2	372	32	0.1	64	0.0001	48	0.000000e+00	1.000000e-08	93.94	95.56	94.63	95.82	93.28	93.09	94.37	94.21	90.55	94.30	92.60	92.95	94.24	94.32	92.88	92.06	93.49	93.30	92.21	91.82	92.93	94.05	94.45	91.26	93.50
3	372	32	0.1	64	0.0001	48	1.000000e-08	1.000000e-08	92.27	95.17	93.79	93.81	94.48	91.81	94.70	92.62	95.45	92.59	93.04	94.49	93.38	93.11	91.70	95.94	93.29	93.75	92.62	93.04	93.96	94.52	93.33	91.51	90.11
0	372	32	0.1	64	0.0010	48	0.000000e+00	1.000000e-08	92.16	92.19	93.56	93.63	94.31	91.97	95.74	92.59	92.73	94.42	91.45	93.78	95.29	93.21	92.46	93.78	93.16	94.59	92.95	92.43	92.80	93.70	94.08	92.28	90.86

Size of the test data:  (6, 33)

Difference

diff_mw = dt_mw[['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
for i, cols in enumerate(zip(dt_mw.columns,test_data_mw1.columns)):   
   diff_mw[f'col_diff_{i}'] = (np.array(dt_mw[cols[0]]) - np.array(test_data_mw1[cols[1]]))*-1

# Function to count negatives in a row
def count_negatives(row):
    return sum(1 for value in row if isinstance(value, (int, float)) and value < 0)

diff_mw['negative_count'] = diff_mw.apply(count_negatives, axis=1)

diff_mw.sort_values(by =["negative_count"],inplace = True, ascending=False)


html_table = diff_mw.to_html(index=True)

# Wrap in a scrollable div
scrollable_table = f"""
<div style="height: 400px; width: 100%; overflow-x: auto; overflow-y: auto;">
    {html_table}
</div>
"""
# Display the scrollable table
display(HTML(scrollable_table))

# show(diff)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	col_diff_0	col_diff_1	col_diff_2	col_diff_3	col_diff_4	col_diff_5	col_diff_6	col_diff_7	col_diff_8	col_diff_9	col_diff_10	col_diff_11	col_diff_12	col_diff_13	col_diff_14	col_diff_15	col_diff_16	col_diff_17	col_diff_18	col_diff_19	col_diff_20	col_diff_21	col_diff_22	col_diff_23	col_diff_24	col_diff_25	col_diff_26	col_diff_27	col_diff_28	col_diff_29	col_diff_30	col_diff_31	col_diff_32	negative_count
1135	372	32	0.1	64	0.0001	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.13	8.26	5.03	9.06	6.05	4.55	10.35	6.48	9.72	5.16	5.33	6.86	6.10	5.26	2.87	8.41	6.65	7.15	2.63	1.83	5.24	3.75	5.55	0.15	-0.42	1
1068	372	32	0.1	32	0.0001	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.27	4.08	7.36	7.29	7.83	5.80	9.00	6.60	5.06	6.63	2.90	5.07	6.24	7.23	3.32	7.25	7.80	6.55	5.06	3.71	4.43	4.13	5.91	4.09	2.77	0
111	372	32	0.1	32	0.0010	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.60	9.86	7.19	6.91	4.45	5.09	6.31	10.03	4.86	6.36	3.93	4.94	5.93	7.35	7.73	8.56	6.82	6.27	4.40	4.28	1.37	6.05	4.12	1.32	2.15	0
495	372	32	0.1	32	0.0010	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.18	10.28	7.32	8.16	5.21	6.81	4.32	6.56	5.92	7.01	7.61	4.45	0.64	7.76	3.34	7.28	6.70	5.75	4.05	1.55	3.42	4.63	0.71	5.35	4.35	0
1123	372	32	0.1	64	0.0001	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.72	10.28	7.23	8.85	7.77	2.74	8.26	5.88	4.29	7.47	5.15	4.57	5.66	7.18	4.43	5.23	6.74	5.26	4.17	3.76	3.40	3.91	4.42	1.30	4.19	0
547	372	32	0.1	64	0.0010	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.02	4.53	7.06	6.62	6.84	2.23	7.76	5.45	5.56	4.91	3.06	4.55	6.41	5.42	6.19	3.54	7.65	9.70	0.77	2.55	4.23	5.32	4.14	0.79	0.52	0

Model performance on valid data

Average & Std Deviation of the Significant rows:

	mean	std
mAP_valid_zero_26	90.036667	0.437066
mAP_valid_zero_25	89.820000	1.464445
mAP_valid_zero_20	89.516667	1.528511
mAP_valid_zero_23	89.256667	1.246718
mAP_valid_zero_21	89.216667	0.903497
mAP_valid_zero_19	89.186667	1.743005
mAP_valid_zero_13	89.043333	1.106520
mAP_valid_zero_22	88.851667	1.698699
mAP_valid_zero_12	88.705000	0.750007
mAP_valid_zero_6	88.296667	1.443089
mAP_valid_zero_10	87.970000	0.909043
mAP_valid_zero_15	87.845000	1.143202
mAP_valid_zero_16	87.840000	1.369584
mAP_valid_zero_11	87.773333	0.746690
mAP_valid_zero_9	87.478333	1.368465
mAP_valid_zero_18	87.238333	1.343658
mAP_valid_zero_8	87.105000	1.023460
mAP_valid_zero_3	87.103333	1.230637
mAP_valid_zero_7	86.980000	1.927278
mAP_valid_zero_14	86.911667	0.996823
mAP_valid_zero_5	86.758333	2.042395
mAP_valid_zero_17	86.543333	0.682984
mAP_valid_zero_2	86.353333	1.214671
mAP_valid_zero	86.236667	1.379763
mAP_valid_zero_4	86.051667	0.913552

Model performance on Test data

We then apply the hyperparameters to the test set and average the results.

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params = dt_valid.loc[dt_valid_sub_mw1.index].iloc[:, :6].reset_index()
optimal_params.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_1 = pd.merge(optimal_params, dt_test, how='left')
test_data_1_sub = test_data_1[list(test_data_1.columns[test_data_1.columns.str.startswith('mAP_test')])]
test_data_1['average_map'] = test_data_1_sub.apply(np.mean, axis=1)

test_data_1.sort_values(by=['average_map'], ascending= False,inplace=True)

	mean	std
mAP_test_zero_3	94.292917	1.041042
mAP_test_zero_13	94.009167	1.373906
mAP_test_zero_10	93.998333	0.755172
mAP_test_zero_4	93.960833	1.190626
mAP_test_zero_8	93.853750	0.970562
mAP_test_zero_17	93.666667	1.033204
mAP_test_zero_14	93.652083	1.329197
mAP_test_zero	93.597917	1.074021
mAP_test_zero_18	93.537083	1.414526
mAP_test_zero_5	93.393750	1.484417
mAP_test_zero_12	93.157083	0.991803
mAP_test_zero_15	93.125417	1.122567
average_map	92.835867	1.194375
mAP_test_zero_11	92.799583	0.748317
mAP_test_zero_2	92.492917	7.298858
mAP_test_zero_26	92.369167	1.310433
mAP_test_zero_19	92.331667	1.237883
mAP_test_zero_6	92.320417	0.990830
mAP_test_zero_7	92.237500	9.436777
mAP_test_zero_20	92.150000	1.382625
mAP_test_zero_9	92.150000	9.466575
mAP_test_zero_16	92.095833	9.453266
mAP_test_zero_23	92.007917	7.157574
mAP_test_zero_22	91.994167	7.105720
mAP_test_zero_25	91.879167	2.305935
mAP_test_zero_21	89.823333	9.499376

Summary using radar plot

res1_valid['id'] = res1_valid.index.to_series().apply(extract_number)
res1_test['id'] = res1_test.index.to_series().apply(extract_number)



res_comb = pd.concat([res1_valid,res1_test])
index_series = res_comb.index.to_series()

res_comb['type'] = np.where(
    index_series.str.contains('valid', case=False), 'valid',
    np.where(index_series.str.contains('test', case=False), 'test', 'unknown')
)
res_comb = res_comb.query("type !='unknown'")

res_comb =  res_comb.reset_index(drop=True)


res1_test = res1_test.sort_values(by=['id']).reset_index().query("index !='average_map'")

data_range1 = np.array(list(res1_valid['mean']) + list(res1_test['mean']))

categories = [str(i) for i in range(1,26)]

fig = go.Figure()

# Valid
fig.add_trace(go.Scatterpolar(
       r=list(res1_test['mean']),
      theta=categories,
      #fill='toself',
      name='Test'
))

# Test
fig.add_trace(go.Scatterpolar(
      r=list(res1_valid['mean']),
      theta=categories,
      #fill='toself',
      name='Valid'
))

# Customization of chart
fig.update_layout(
  title={
      'text': "T-Test",
        'xanchor': 'center',
      'yanchor': 'top'
  },
  polar=dict(
    radialaxis=dict(
      visible=True,
      range=[round(data_range1.min()), round(data_range1.max()) + 1]
    )),
  showlegend=True
)

fig.show()




##############


res2_valid['id'] = res2_valid.index.to_series().apply(extract_number)
res2_test['id'] = res2_test.index.to_series().apply(extract_number)


res_comb = pd.concat([res2_valid,res2_test])
index_series = res_comb.index.to_series()

res_comb['type'] = np.where(
    index_series.str.contains('valid', case=False), 'valid',
    np.where(index_series.str.contains('test', case=False), 'test', 'unknown')
)
res_comb = res_comb.query("type !='unknown'")

res_comb =  res_comb.reset_index(drop=True)


res2_test = res2_test.sort_values(by=['id']).reset_index().query("index !='average_map'")

data_range2 = np.array(list(res2_valid['mean']) + list(res2_test['mean']))

categories = [str(i) for i in range(1,26)]
fig = go.Figure()

# Valid
fig.add_trace(go.Scatterpolar(
       r=list(res2_test['mean']),
      theta=categories,
      #fill='toself',
      name='Test'
))

# Test
fig.add_trace(go.Scatterpolar(
      r=list(res2_valid['mean']),
      theta=categories,
      #fill='toself',
      name='Valid'
))

# Customization of chart
fig.update_layout(
  title={
      'text': "MU-Test",
         'xanchor': 'center',
      'yanchor': 'top'
  },
  polar=dict(
    radialaxis=dict(
      visible=True,
      range=[round(data_range2.min()), round(data_range2.max()) + 1]
    )),
  showlegend=True
)

fig.show()

best_valid_bit_size_32 = round(res1_valid['mean'][0])
id_best_valid = res1_valid['id'][0]
best_test_bit_size_32 = list(round(res1_test.query('id == @id_best_valid')['mean'],2))[0]

best_valid_bit_size_32_mw = round(res2_valid['mean'][0])
id_best_valid = res2_valid['id'][0]
best_test_bit_size_32_mw = list(round(res2_test.query('id == @id_best_valid')['mean'],2))[0]

Bit size 48

The results in this presentation are from two experimental designs:

Experiment 1 Results

The thresholding is based on fixed values between -1 and 1 on a step size of 0.1.

Validation Data mAP Results Preview:

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_abs_values	mAP_valid_abs_values_2	mAP_valid_abs_values_3	mAP_valid_abs_values_4	mAP_valid_abs_values_5	mAP_valid_abs_values_6	mAP_valid_abs_values_7	mAP_valid_abs_values_8	mAP_valid_abs_values_9	mAP_valid_abs_values_10	mAP_valid_abs_values_11	mAP_valid_abs_values_12	mAP_valid_abs_values_13	mAP_valid_abs_values_14	mAP_valid_abs_values_15	mAP_valid_abs_values_16	mAP_valid_abs_values_17	mAP_valid_abs_values_18	mAP_valid_abs_values_19	mAP_valid_abs_values_20	mAP_valid_abs_values_21	mAP_valid_abs_values_22	mAP_valid_abs_values_23	mAP_valid_abs_values_25	mAP_valid_abs_values_26
48	372	48	0.1	16	0.00100	24	0.000000e+00	0.000000e+00	84.37	84.42	87.65	85.57	85.97	86.24	84.97	88.17	83.80	86.06	85.92	87.89	86.91	85.17	85.75	87.81	83.44	86.14	88.83	90.70	88.11	85.70	88.93	87.04	87.10
51	372	48	0.1	16	0.00100	24	0.000000e+00	1.000000e-08	84.65	87.42	85.04	89.28	85.13	87.16	91.29	84.61	87.99	85.90	87.84	88.38	88.23	84.18	83.91	85.59	87.67	88.54	87.67	88.78	89.99	90.23	89.27	89.16	87.51
60	372	48	0.1	16	0.00100	24	1.000000e-08	0.000000e+00	85.17	86.18	85.72	86.21	83.17	43.72	84.13	89.31	84.94	87.89	88.82	86.67	43.72	43.72	86.86	87.49	86.28	43.72	88.68	87.84	89.11	89.32	87.60	88.61	89.40
63	372	48	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	84.67	86.49	85.08	85.98	85.41	88.27	85.98	88.78	85.22	84.68	88.41	87.46	86.96	87.65	85.50	88.03	86.39	88.08	86.94	89.72	88.25	86.24	88.45	87.68	89.20
112	372	48	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	85.47	85.63	85.61	84.77	87.69	87.58	82.77	87.41	86.44	84.13	87.81	87.61	87.27	86.71	87.61	88.18	85.11	87.53	88.37	86.91	89.44	88.66	88.36	87.82	88.74
115	372	48	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	84.12	86.18	86.90	84.79	86.35	87.68	85.68	87.64	86.02	85.78	89.89	88.64	87.55	85.76	86.52	85.93	86.84	85.93	87.27	90.80	89.44	87.55	89.97	88.68	88.75
124	372	48	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	86.28	87.33	86.70	86.02	85.89	87.03	85.87	86.91	87.73	86.03	87.93	88.10	88.20	88.13	88.07	87.56	87.53	88.51	88.52	88.84	89.75	89.07	89.94	89.01	88.74
127	372	48	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	84.52	85.82	86.31	86.99	87.74	87.99	86.04	86.08	88.75	87.09	87.63	90.45	88.72	87.64	90.28	87.88	87.44	86.95	88.64	88.30	88.01	89.05	88.02	88.49	88.94
176	372	48	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	87.87	87.39	43.72	84.70	83.73	86.66	85.94	87.61	86.84	88.05	87.53	88.44	86.92	85.56	87.26	86.79	86.61	87.26	87.64	87.19	89.46	89.24	85.59	88.02	87.31
179	372	48	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	84.86	87.46	85.43	83.89	85.79	88.82	84.40	85.62	87.25	86.17	86.97	88.66	88.08	87.32	88.11	86.80	86.66	84.83	89.20	90.28	88.88	87.27	90.56	88.52	89.05
188	372	48	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	84.51	88.86	86.31	85.30	87.49	88.14	87.27	88.62	87.45	88.43	88.58	87.60	87.37	83.51	84.52	89.82	85.67	87.12	88.59	90.26	89.54	88.72	90.84	88.24	88.59
191	372	48	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	86.43	87.74	85.72	85.02	88.02	87.65	85.25	84.59	87.20	84.71	87.77	87.16	87.87	87.48	85.76	87.60	84.85	86.86	87.50	90.19	86.33	88.74	88.38	88.66	90.41
240	372	48	0.1	16	0.00100	36	0.000000e+00	0.000000e+00	84.46	43.72	83.77	88.53	85.61	86.67	83.59	84.95	84.20	84.44	88.39	88.07	86.11	85.86	85.29	87.55	86.88	87.64	85.54	88.40	89.06	86.94	88.79	89.28	88.14
243	372	48	0.1	16	0.00100	36	0.000000e+00	1.000000e-08	83.20	86.35	87.36	87.45	84.73	86.62	86.00	86.41	85.94	84.50	85.76	86.86	88.17	84.81	87.18	85.66	86.49	85.93	87.88	91.15	87.69	88.41	88.37	86.65	88.39
252	372	48	0.1	16	0.00100	36	1.000000e-08	0.000000e+00	84.28	85.44	84.74	85.81	88.41	87.48	86.25	87.56	85.02	84.65	90.86	88.80	88.59	87.40	88.53	86.04	86.44	88.08	88.34	87.91	88.60	89.09	88.49	87.75	88.67
255	372	48	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	85.90	84.85	88.51	87.27	85.17	87.42	85.79	85.81	87.29	87.03	86.66	89.80	86.03	84.64	86.47	85.59	87.10	86.81	87.82	88.22	89.01	89.31	88.90	87.46	88.85
304	372	48	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	85.12	85.65	85.23	87.09	86.68	87.95	86.19	83.69	85.19	86.51	89.27	88.83	88.69	88.46	85.59	87.14	86.13	86.33	89.74	87.74	88.89	88.34	88.66	88.34	88.48
307	372	48	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	87.32	88.00	86.61	86.65	86.67	89.92	88.18	87.08	89.73	86.18	87.98	89.14	87.43	87.64	87.01	89.51	86.58	88.45	89.12	89.51	89.09	87.84	88.45	89.98	86.23
316	372	48	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	87.97	87.43	86.48	86.58	87.77	86.80	85.76	85.76	87.22	88.65	87.76	87.33	88.35	87.87	87.71	86.89	86.82	87.85	92.34	89.92	89.74	89.21	89.32	91.17	91.82
319	372	48	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	86.55	86.72	86.63	87.15	86.06	88.33	89.32	86.37	88.37	86.80	89.27	88.61	87.47	87.79	86.53	85.97	87.84	85.55	88.32	89.40	88.20	90.20	88.40	89.82	88.98
368	372	48	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	88.72	85.60	85.77	87.74	85.21	88.17	86.37	85.54	87.18	87.84	87.98	87.63	88.54	86.94	87.19	87.53	85.43	88.37	89.47	88.03	89.34	90.30	87.79	88.73	87.73
371	372	48	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	86.06	85.50	83.72	86.92	85.78	85.51	86.55	87.23	85.83	86.05	88.27	87.64	86.62	85.18	87.26	87.41	87.37	85.45	88.10	87.08	87.98	90.30	88.21	88.99	88.87
380	372	48	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	85.92	86.66	86.61	85.23	90.32	88.00	85.00	85.46	88.20	88.34	86.89	87.83	84.98	84.23	86.50	86.74	89.95	87.70	88.34	88.52	90.04	89.48	87.08	88.65	88.87
383	372	48	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	85.37	86.79	86.23	86.44	88.35	87.11	87.44	86.02	86.08	86.97	88.52	87.53	85.87	88.20	84.72	86.85	87.27	87.64	90.73	86.81	88.46	86.09	89.51	89.17	86.14
432	372	48	0.1	16	0.00100	48	0.000000e+00	0.000000e+00	84.51	86.40	88.47	84.76	83.46	84.53	85.02	85.37	85.15	85.00	87.69	87.68	86.46	85.48	88.48	87.11	85.62	85.93	87.54	87.72	90.44	87.46	89.35	87.96	87.62
435	372	48	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	85.31	87.36	87.02	85.20	88.81	86.82	87.76	85.98	88.76	88.09	91.31	88.19	88.26	85.73	87.49	88.29	86.44	88.29	90.18	87.33	90.03	89.41	89.05	87.39	88.70
444	372	48	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	85.01	87.24	86.60	89.07	86.15	86.83	84.85	86.06	86.97	85.76	87.65	87.06	88.70	86.80	86.51	86.08	87.10	85.34	43.72	87.48	88.60	90.62	89.13	43.72	88.22
447	372	48	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	85.33	86.12	85.39	86.57	86.06	85.50	84.64	86.34	86.34	43.72	86.65	88.80	84.68	86.91	86.51	87.88	87.12	88.50	87.44	43.72	87.12	90.15	88.67	91.01	88.19
496	372	48	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	85.04	85.58	84.64	84.91	86.96	88.16	85.27	86.54	88.34	86.25	87.84	88.85	89.02	87.41	83.31	87.61	88.33	86.46	88.24	89.97	89.98	88.70	89.74	90.38	88.58
499	372	48	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	86.99	86.54	85.61	83.16	85.29	87.93	86.92	86.87	86.83	87.80	89.99	89.87	86.64	84.94	87.18	87.86	85.30	87.05	89.13	88.90	89.41	89.38	89.58	89.54	86.34
508	372	48	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	88.36	86.85	87.99	87.14	86.38	87.14	87.33	86.59	86.02	86.24	90.63	89.15	89.56	86.25	86.79	87.56	85.76	86.38	88.67	89.39	89.22	88.00	87.90	88.63	90.68
511	372	48	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	85.95	86.62	86.10	89.47	85.59	87.96	86.92	87.15	87.14	84.29	86.22	89.45	86.61	86.09	88.10	86.47	87.15	86.61	89.32	88.44	88.00	89.30	86.57	87.69	85.67
560	372	48	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	86.09	87.81	85.84	81.76	83.75	88.07	86.36	87.67	87.48	85.58	86.99	88.11	86.10	84.79	89.68	90.14	88.42	86.08	85.12	86.36	89.52	89.80	87.54	89.39	87.55
563	372	48	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	85.04	84.99	85.10	87.88	86.60	87.23	83.26	88.52	85.91	87.84	89.66	87.90	88.13	90.54	87.26	86.75	87.27	85.58	87.40	85.56	89.79	89.31	87.79	89.25	87.70
572	372	48	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	87.81	83.83	85.38	86.63	86.87	88.14	85.20	87.55	86.54	86.38	86.38	88.40	87.21	89.93	88.13	84.97	85.02	85.70	90.95	88.25	90.45	89.46	88.41	87.86	89.00
575	372	48	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	84.77	84.87	86.62	87.27	84.10	85.01	87.10	85.49	86.73	84.21	87.29	85.09	86.63	89.80	89.73	86.79	88.82	87.61	87.12	89.50	88.26	87.53	86.83	89.91	89.13
624	372	48	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	83.05	84.70	83.84	88.62	84.72	86.85	84.05	87.59	84.59	84.78	91.24	86.87	86.26	86.21	87.05	84.02	85.69	87.18	88.76	88.29	88.60	87.49	88.70	88.73	86.64
627	372	48	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	83.73	85.65	85.84	86.99	87.43	86.16	86.02	87.86	87.00	85.97	87.11	86.31	87.94	86.96	87.80	87.18	86.58	87.02	89.01	88.31	89.46	88.50	88.69	89.13	87.62
636	372	48	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	86.27	85.77	85.20	84.63	85.31	86.24	84.68	86.26	85.49	84.22	89.64	87.64	86.41	88.10	86.35	87.93	86.24	87.66	88.95	88.18	89.88	89.23	90.55	85.83	89.66
639	372	48	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	85.50	85.89	85.41	86.29	84.69	85.50	82.74	85.73	84.36	85.00	85.29	87.27	87.60	88.53	87.74	84.62	88.52	86.40	88.11	86.77	88.29	87.48	88.52	88.24	87.21
688	372	48	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	85.08	88.19	86.71	85.80	83.08	86.50	86.85	85.73	89.97	89.19	87.52	87.64	87.18	86.73	88.27	88.88	84.19	87.51	88.80	87.71	88.44	88.39	87.69	89.14	87.26
691	372	48	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	87.37	86.62	87.45	86.35	88.49	89.31	86.41	90.66	87.17	85.77	89.63	88.47	87.05	88.37	87.99	87.58	86.36	86.24	90.28	89.40	88.24	93.05	89.50	90.33	89.35
700	372	48	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	86.83	86.92	86.94	86.14	85.74	86.88	85.93	86.62	87.20	88.25	90.36	86.18	86.23	89.14	86.58	87.65	85.80	87.19	89.48	88.04	88.21	88.63	89.25	89.88	88.88
703	372	48	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	87.54	87.53	87.10	87.99	86.91	86.16	90.15	86.02	88.54	87.97	88.59	87.70	86.10	85.65	87.19	87.16	85.86	84.95	89.60	89.75	88.66	89.09	88.28	88.05	89.55
752	372	48	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	85.57	84.99	84.71	82.51	82.91	86.61	87.50	86.39	85.17	85.35	86.76	87.51	88.66	87.29	86.70	85.17	85.33	87.92	86.49	89.74	89.05	89.05	86.93	87.63	87.29
755	372	48	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	87.15	89.30	86.12	86.29	88.10	87.95	87.62	89.40	88.18	85.85	89.19	89.72	89.27	87.64	85.49	85.87	87.27	87.85	89.46	89.46	90.70	89.08	87.53	89.91	90.16
764	372	48	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	86.12	85.42	86.33	84.78	88.09	87.22	87.28	87.74	85.96	86.23	88.16	87.13	86.26	86.46	88.10	88.81	89.16	87.10	89.09	89.77	90.97	88.25	89.53	89.59	88.09
767	372	48	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	85.86	85.33	86.08	85.98	83.76	87.78	84.47	87.02	87.35	87.72	87.75	86.65	86.19	87.94	86.27	86.14	85.91	85.51	87.57	87.89	88.46	88.38	89.50	89.85	87.65
816	372	48	0.1	16	0.00010	36	0.000000e+00	0.000000e+00	84.36	84.02	84.57	87.47	85.15	85.02	86.34	85.41	86.41	85.90	85.99	88.65	87.32	88.63	88.75	87.31	86.70	87.08	90.06	87.93	88.67	87.40	88.91	87.91	87.37
819	372	48	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	84.36	87.86	85.14	85.43	84.97	89.72	84.86	85.11	85.00	85.77	86.41	87.04	85.82	86.23	87.94	87.28	85.87	87.26	89.45	87.46	88.38	88.79	87.67	88.68	87.86
828	372	48	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	84.71	85.82	86.69	85.40	84.41	85.97	83.40	88.34	86.22	85.55	89.19	88.67	87.75	87.90	86.67	87.44	86.60	86.44	90.53	90.61	88.98	90.52	88.76	89.39	87.84
831	372	48	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	84.89	85.57	84.12	87.58	86.37	87.47	85.60	87.97	85.38	85.88	88.36	86.09	88.08	86.74	89.27	86.85	86.32	84.74	89.76	87.66	87.81	88.76	88.69	88.45	88.46
880	372	48	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	87.46	85.24	85.72	84.48	86.71	87.90	83.03	86.56	89.26	85.85	88.76	86.00	84.68	85.00	87.60	89.52	87.10	87.67	88.30	87.73	88.88	87.08	87.37	89.04	89.72
883	372	48	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	88.07	43.72	88.33	87.20	86.55	88.21	86.29	87.40	87.02	87.62	86.36	89.27	85.97	84.64	85.97	88.16	87.88	87.46	87.72	87.96	87.05	43.72	89.48	87.88	88.43
892	372	48	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	87.05	87.43	87.24	85.47	86.21	88.10	86.70	88.84	88.14	87.37	89.02	87.30	88.20	86.63	86.45	88.11	86.91	86.36	88.77	89.88	88.52	88.89	89.66	88.68	89.26
895	372	48	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	85.61	88.38	85.63	87.49	87.86	87.52	85.12	86.15	86.89	87.75	88.74	87.45	89.02	87.14	87.37	88.88	86.57	87.43	88.44	89.52	89.36	89.52	89.02	90.31	88.79
944	372	48	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	84.95	85.42	89.58	85.96	86.85	87.58	85.58	88.36	87.00	87.50	87.48	86.34	88.44	86.64	86.41	87.80	87.20	85.54	87.89	87.71	90.06	88.93	88.92	89.84	88.10
947	372	48	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	87.03	86.18	87.22	88.15	85.78	86.80	87.65	88.55	86.23	86.79	87.56	87.14	86.49	84.95	88.11	84.86	86.21	86.37	87.36	89.67	90.61	88.74	88.60	87.05	88.93
956	372	48	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	87.37	86.60	86.20	85.30	86.01	86.60	88.60	88.63	87.27	89.30	88.91	89.14	85.91	87.57	86.11	87.39	86.08	86.88	88.80	87.47	86.84	89.41	87.78	87.90	87.21
959	372	48	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	86.09	85.02	86.14	86.08	86.35	88.06	86.17	86.27	86.05	87.38	86.54	89.28	87.35	85.70	90.59	87.77	83.68	87.06	89.17	88.56	90.30	89.12	88.87	86.65	88.18
1008	372	48	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	85.15	86.48	85.61	86.40	84.01	87.14	86.54	86.08	87.68	86.06	88.30	91.42	87.89	86.24	88.14	87.69	86.93	87.95	90.18	88.18	89.10	88.27	87.06	89.62	86.04
1011	372	48	0.1	16	0.00010	48	0.000000e+00	1.000000e-08	86.04	87.35	86.65	85.12	84.76	88.26	85.97	86.47	86.84	86.10	86.38	84.70	86.61	84.24	85.17	86.73	86.67	86.41	88.00	88.04	88.43	89.63	88.78	90.01	88.08
1020	372	48	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	85.60	87.07	86.35	90.41	85.34	87.96	85.23	87.80	86.79	89.02	88.58	87.93	86.76	85.46	86.77	87.08	87.41	87.66	88.73	89.30	89.08	89.63	88.91	89.66	89.09
1023	372	48	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	84.33	86.79	87.50	88.68	85.99	90.55	88.03	86.45	87.50	88.57	87.07	89.91	87.59	87.17	86.97	88.85	88.96	86.76	88.03	87.13	89.36	88.05	90.22	87.84	87.12
1072	372	48	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	83.53	84.73	87.18	87.12	88.77	87.70	85.51	87.19	86.87	87.82	87.71	88.13	87.78	86.70	85.34	85.73	85.44	86.75	89.10	90.45	89.93	87.54	87.95	89.17	88.75
1075	372	48	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	86.26	85.85	84.84	86.26	88.01	87.98	87.68	86.68	88.53	86.34	87.79	88.27	86.22	86.80	86.06	88.91	85.12	86.67	89.90	88.34	89.77	88.40	87.84	87.58	89.55
1084	372	48	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	88.63	88.33	87.15	86.58	89.23	88.69	88.24	87.40	89.00	86.43	87.32	89.68	87.52	88.24	88.39	89.32	85.88	87.85	88.06	89.80	89.57	88.35	87.96	89.94	90.95
1087	372	48	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	86.25	85.62	85.75	85.22	85.81	85.45	86.13	86.51	87.22	85.84	89.21	87.02	88.03	86.46	86.58	88.04	86.46	86.63	88.43	89.74	89.52	88.51	87.68	87.98	88.47
1136	372	48	0.1	64	0.00010	48	0.000000e+00	0.000000e+00	83.47	86.75	85.57	86.01	86.37	87.40	85.58	87.94	84.60	86.86	88.62	84.94	87.08	88.28	85.16	88.70	87.89	87.20	87.70	86.76	80.84	89.24	88.68	89.15	89.91
1139	372	48	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	87.72	84.12	86.73	84.93	86.78	88.55	83.86	88.54	87.94	86.19	86.27	85.77	84.95	87.96	86.55	89.47	86.41	85.77	87.02	87.95	85.83	88.20	88.47	88.36	87.31
1148	372	48	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	86.43	86.40	86.17	84.73	82.01	84.73	85.71	85.16	88.02	86.65	89.17	89.33	84.82	84.24	84.70	87.00	86.30	85.23	87.22	87.66	87.50	90.76	87.77	90.48	87.81
1151	372	48	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	86.12	86.48	86.93	88.27	84.68	87.09	87.67	89.66	86.65	87.59	87.84	88.95	88.64	86.36	86.17	87.12	85.48	86.99	87.89	88.08	89.58	89.54	89.82	89.26	88.31
1200	372	48	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	84.90	85.14	84.62	89.46	86.80	86.42	84.27	86.51	85.60	84.20	87.79	88.01	85.06	85.63	86.10	85.76	86.94	85.40	87.94	87.24	89.25	89.08	87.66	87.89	87.83
1203	372	48	0.1	16	0.00001	24	0.000000e+00	1.000000e-08	85.27	86.46	85.89	88.30	86.35	86.84	83.43	86.38	87.09	84.70	87.54	88.71	86.33	86.29	88.93	87.11	87.61	86.55	87.78	89.74	89.06	88.76	88.40	87.78	89.36
1212	372	48	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	85.84	87.15	85.72	86.99	85.41	86.96	85.94	87.46	88.25	85.67	89.45	87.81	86.63	86.33	86.62	86.05	86.86	85.80	88.13	89.72	88.44	87.37	88.47	88.98	88.43
1215	372	48	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	85.49	85.56	85.98	85.17	84.33	85.25	83.02	88.34	84.44	84.82	87.18	88.45	89.29	86.30	86.43	85.89	86.59	86.73	86.88	86.63	89.42	86.02	88.02	88.55	88.80
1264	372	48	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	85.46	86.87	84.17	84.40	87.41	88.03	83.49	89.10	87.16	87.79	87.60	88.36	86.54	86.21	85.88	88.16	87.52	86.49	88.10	90.70	90.11	88.61	86.91	88.76	89.37
1267	372	48	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	85.03	86.24	86.07	86.98	87.28	86.84	86.90	85.85	86.61	88.16	88.75	87.78	87.06	89.04	87.60	85.37	88.43	86.77	88.10	88.62	89.63	90.12	89.48	88.97	89.07
1276	372	48	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	85.77	85.66	86.80	87.11	87.92	86.57	87.60	87.04	86.05	85.70	89.08	87.74	86.80	86.59	86.92	86.85	88.12	89.60	88.65	87.62	89.62	88.86	86.21	89.44	91.27
1279	372	48	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	86.25	88.00	85.87	84.34	87.64	88.13	86.68	85.15	87.88	86.87	87.75	88.27	87.50	84.66	87.62	87.38	87.51	85.29	88.81	88.20	89.08	89.29	89.27	88.87	89.38
1328	372	48	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	85.37	88.49	86.96	62.09	86.75	86.69	87.16	84.87	87.76	85.66	86.82	89.57	88.30	87.45	87.43	87.28	87.28	84.63	87.78	88.95	87.94	88.28	89.42	86.09	83.52
1331	372	48	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	86.30	84.51	86.39	85.87	86.59	86.70	85.90	85.50	91.28	86.84	87.32	87.86	87.92	87.03	87.35	87.70	87.00	87.01	88.53	88.61	89.22	87.05	88.47	89.03	88.92
1340	372	48	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	85.97	85.67	85.28	87.44	85.37	88.08	85.43	89.58	85.91	86.83	86.50	87.22	86.56	88.91	87.92	88.68	86.06	86.59	89.28	88.15	89.80	87.90	86.29	89.23	87.47
1343	372	48	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	84.44	87.23	86.07	87.72	86.62	88.09	86.78	88.11	85.99	86.84	90.48	88.72	89.36	87.62	85.15	87.53	84.99	86.56	87.94	88.46	89.38	89.17	87.75	88.93	89.26
1392	372	48	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	84.15	87.22	86.42	86.33	86.04	86.82	87.21	89.18	89.37	85.79	86.46	88.47	87.49	86.19	88.18	87.74	87.13	86.39	87.37	87.13	90.22	89.20	89.60	89.99	90.32
1395	372	48	0.1	16	0.00001	36	0.000000e+00	1.000000e-08	84.90	87.43	86.06	88.63	84.69	87.60	85.34	85.84	84.66	86.61	88.77	88.96	87.36	83.22	86.62	86.52	85.22	86.92	90.55	88.74	88.30	89.61	88.27	89.12	88.43
1404	372	48	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	86.09	84.80	86.33	89.11	84.81	87.82	85.70	85.69	85.95	86.25	88.26	87.35	86.34	86.18	86.19	88.62	85.09	89.40	87.06	88.53	87.93	87.74	89.14	88.35	87.80
1407	372	48	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	84.54	86.02	86.79	84.80	84.56	86.47	86.58	90.43	85.35	85.22	88.05	88.21	87.34	87.74	90.88	87.82	87.15	86.11	87.67	87.38	90.00	89.06	89.84	87.86	90.23
1456	372	48	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	88.13	88.06	85.81	86.94	84.89	84.96	85.72	90.37	84.09	85.97	87.62	89.66	86.74	86.67	85.44	86.64	85.54	86.49	88.80	89.12	88.56	89.69	88.76	90.50	87.33
1459	372	48	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	86.47	87.77	87.14	85.97	86.59	86.68	86.36	86.79	86.10	87.53	89.54	86.05	86.22	87.83	85.83	86.73	85.96	87.43	87.78	87.72	90.11	89.91	91.01	88.81	89.62
1468	372	48	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	87.90	86.66	86.71	85.40	87.97	89.21	86.53	84.85	88.66	86.64	88.41	87.14	87.23	87.33	87.33	88.78	88.90	86.67	91.27	87.59	89.03	88.67	89.27	88.59	89.84
1471	372	48	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	86.25	86.75	87.45	90.66	84.77	86.47	88.10	86.77	84.65	86.68	88.72	89.57	89.73	83.16	86.04	88.15	85.51	89.01	88.81	91.07	89.49	89.02	90.18	88.63	87.06
1520	372	48	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	85.42	84.39	85.96	87.60	86.58	88.40	86.99	87.23	86.26	87.82	89.02	87.02	87.56	84.89	86.15	88.15	88.18	87.59	87.49	88.21	89.23	87.80	88.26	89.92	89.07
1523	372	48	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	87.12	85.91	85.31	87.08	90.59	85.70	86.90	86.19	85.99	86.88	86.40	91.51	88.11	88.10	86.45	86.53	84.76	86.15	91.44	88.36	86.39	88.99	89.52	89.48	87.58
1532	372	48	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	85.27	87.21	85.21	85.60	86.48	87.50	84.60	88.30	85.58	87.16	87.38	87.89	85.67	88.23	84.52	89.38	86.06	86.81	87.51	89.28	89.17	87.09	89.28	87.55	88.41
1535	372	48	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	85.43	87.50	89.00	86.80	86.12	86.31	86.95	85.54	86.01	86.10	87.92	88.16	87.81	87.67	86.52	87.75	90.14	87.07	89.75	92.23	88.93	88.88	88.37	88.85	88.22

Test Data mAP Results Preview:

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_abs_values	mAP_test_abs_values_2	mAP_test_abs_values_3	mAP_test_abs_values_4	mAP_test_abs_values_5	mAP_test_abs_values_6	mAP_test_abs_values_7	mAP_test_abs_values_8	mAP_test_abs_values_9	mAP_test_abs_values_10	mAP_test_abs_values_11	mAP_test_abs_values_12	mAP_test_abs_values_13	mAP_test_abs_values_14	mAP_test_abs_values_15	mAP_test_abs_values_16	mAP_test_abs_values_17	mAP_test_abs_values_18	mAP_test_abs_values_19	mAP_test_abs_values_20	mAP_test_abs_values_21	mAP_test_abs_values_22	mAP_test_abs_values_23	mAP_test_abs_values_25	mAP_test_abs_values_26
48	372	48	0.1	16	0.00100	24	0.000000e+00	0.000000e+00	91.58	91.62	90.60	91.84	92.42	91.43	89.10	90.55	93.86	91.29	91.54	90.19	92.37	93.31	91.47	80.21	92.45	91.87	89.73	89.60	92.03	91.42	90.07	92.53	89.79
51	372	48	0.1	16	0.00100	24	0.000000e+00	1.000000e-08	93.70	92.93	96.39	91.25	94.47	92.51	91.80	94.66	94.78	93.42	91.22	92.95	93.33	94.01	92.98	92.71	94.84	80.75	94.13	90.11	94.24	92.75	91.88	92.27	92.48
60	372	48	0.1	16	0.00100	24	1.000000e-08	0.000000e+00	92.00	93.03	93.99	92.90	94.03	48.12	94.01	93.48	93.17	92.60	93.85	88.37	48.12	48.12	93.08	93.65	92.05	48.12	94.11	91.85	92.71	92.83	90.16	90.56	93.19
63	372	48	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	90.60	92.37	95.02	92.79	91.06	92.18	92.92	92.98	91.45	92.75	91.85	90.47	93.41	91.61	93.15	93.40	92.82	93.95	94.36	92.70	92.59	93.25	92.85	91.94	87.51
112	372	48	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	92.44	94.38	95.75	90.82	93.68	92.79	94.79	94.09	94.57	94.69	93.86	91.75	93.69	93.47	94.38	93.29	94.42	92.92	92.96	93.74	93.24	92.23	91.82	92.55	92.65
115	372	48	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	92.03	92.70	93.99	94.70	90.91	92.70	92.25	91.89	93.81	92.53	83.87	93.75	92.15	92.85	92.80	93.54	94.35	94.14	93.52	90.31	92.20	92.19	92.47	93.70	91.60
124	372	48	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	90.62	94.13	91.43	93.95	94.54	91.67	93.74	94.93	92.50	93.49	89.82	91.57	94.90	93.45	92.72	92.39	93.24	93.59	93.62	91.31	91.73	91.70	91.81	93.05	93.10
127	372	48	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	92.52	92.37	94.59	93.73	95.07	92.57	93.22	92.69	95.96	93.10	94.14	90.79	94.15	96.09	89.53	93.70	94.96	93.76	94.41	92.78	92.75	92.36	89.65	91.54	89.07
176	372	48	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	93.70	90.95	48.12	92.95	93.50	90.97	93.76	95.68	93.71	95.25	94.35	92.90	93.59	95.18	93.97	93.11	91.94	93.33	92.56	92.84	89.16	93.13	91.27	92.25	91.75
179	372	48	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	93.66	92.32	94.43	94.71	92.63	93.39	92.24	93.39	93.47	94.76	89.34	94.43	95.83	95.05	93.45	93.78	95.34	92.45	92.73	91.63	91.97	93.52	94.42	91.17	93.80
188	372	48	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	93.94	92.61	92.99	91.60	93.62	92.79	94.98	94.40	93.57	94.08	91.30	89.75	94.38	91.24	94.74	93.57	95.51	94.78	93.46	89.55	93.61	91.50	94.72	92.09	88.73
191	372	48	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	95.42	95.80	93.76	94.71	95.14	93.02	94.69	92.30	94.45	92.94	94.54	92.14	93.05	94.54	95.92	91.75	95.28	92.56	91.51	90.64	93.44	92.09	93.72	92.65	92.95
240	372	48	0.1	16	0.00100	36	0.000000e+00	0.000000e+00	90.96	48.12	94.03	93.08	91.64	90.96	91.65	91.53	91.86	91.29	91.76	91.56	92.88	93.73	92.81	92.75	90.85	93.92	90.00	92.52	91.03	90.96	94.01	91.25	89.03
243	372	48	0.1	16	0.00100	36	0.000000e+00	1.000000e-08	93.81	91.79	93.14	93.99	93.20	92.05	94.15	93.86	92.52	92.82	92.72	89.72	93.65	92.38	93.96	94.18	93.29	94.93	91.81	91.19	92.14	93.25	94.13	95.56	92.25
252	372	48	0.1	16	0.00100	36	1.000000e-08	0.000000e+00	92.25	95.66	94.36	95.33	95.86	92.08	92.57	95.31	94.64	93.91	94.39	91.82	93.59	94.23	94.43	94.67	94.46	95.45	92.45	91.54	94.64	92.98	93.86	93.87	93.68
255	372	48	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	92.15	91.41	93.38	94.47	94.86	93.85	91.40	94.88	93.27	93.70	93.96	93.50	94.18	94.24	94.79	94.56	95.88	95.06	93.59	91.80	92.36	91.66	93.50	94.81	89.76
304	372	48	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	94.39	91.86	95.35	94.40	93.03	93.89	94.48	95.41	94.43	94.78	92.87	90.67	94.49	93.65	91.14	93.81	92.21	95.02	90.78	91.64	92.54	94.28	93.36	92.92	90.10
307	372	48	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	93.71	94.56	90.96	94.57	92.55	89.98	95.28	94.57	94.32	93.75	93.04	90.96	95.36	93.09	92.27	91.81	95.07	94.65	90.15	92.35	93.34	91.22	93.76	91.11	91.68
316	372	48	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	93.60	95.85	93.82	95.20	93.83	91.42	96.37	93.65	94.94	93.29	92.14	94.10	92.97	94.34	92.59	93.16	95.08	92.05	92.36	91.99	93.15	92.27	92.37	93.10	92.66
319	372	48	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	93.95	91.41	90.78	95.00	94.22	91.58	94.07	94.24	93.00	93.35	92.59	93.98	94.57	94.77	94.41	95.55	91.64	91.90	92.53	93.35	93.70	93.71	92.50	92.18	91.75
368	372	48	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	92.53	93.44	94.70	92.84	95.76	92.27	94.68	95.18	94.72	94.23	94.90	93.61	94.64	93.79	94.13	93.22	94.98	94.27	93.28	93.46	91.82	92.08	92.69	93.92	92.03
371	372	48	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	93.98	93.96	94.47	95.43	95.80	91.97	93.27	94.81	95.34	94.66	94.09	93.52	94.16	90.60	94.27	94.04	95.23	93.80	93.41	92.58	94.69	92.20	94.59	95.00	91.01
380	372	48	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	94.19	92.43	93.25	93.59	92.99	92.56	94.36	94.41	93.87	93.41	91.64	92.36	96.94	97.09	92.70	93.87	93.60	95.72	92.72	92.77	92.08	90.59	93.41	92.03	91.17
383	372	48	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	93.60	94.87	92.87	92.39	95.03	92.02	95.61	93.62	93.62	95.40	92.59	92.94	92.18	95.36	94.81	93.19	94.01	92.38	90.71	91.57	94.58	93.22	91.92	93.74	93.86
432	372	48	0.1	16	0.00100	48	0.000000e+00	0.000000e+00	91.07	94.71	92.42	94.42	95.04	92.94	94.57	91.32	93.05	96.23	91.71	93.55	94.17	94.79	94.08	82.59	92.58	92.63	93.91	89.93	88.70	93.70	91.83	91.76	91.87
435	372	48	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	93.23	95.25	92.71	93.39	93.96	91.78	92.47	93.87	94.74	94.94	91.90	94.74	93.30	92.93	94.05	95.55	94.67	93.63	92.65	89.88	94.25	92.06	92.78	91.43	93.27
444	372	48	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	93.74	94.99	93.71	94.27	94.21	90.25	93.48	95.96	94.52	93.09	93.13	91.27	94.34	94.68	94.15	92.87	95.54	94.53	48.12	92.69	93.11	91.15	93.79	48.12	92.48
447	372	48	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	94.66	94.75	95.28	94.68	95.47	92.66	92.99	95.54	94.74	48.12	93.80	93.64	95.51	94.56	96.34	93.96	96.49	94.30	93.97	48.12	94.90	91.73	92.46	92.90	91.84
496	372	48	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	92.61	92.80	93.05	92.67	94.01	92.72	92.56	94.55	94.03	93.62	91.08	91.52	93.37	93.29	93.92	93.75	95.30	94.09	93.52	92.41	93.15	93.93	94.48	93.48	93.50
499	372	48	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	94.34	95.78	94.15	93.46	93.07	92.47	91.65	94.64	94.42	93.42	84.41	93.52	92.33	94.38	95.28	92.35	95.92	92.18	91.93	91.92	92.33	88.09	93.65	93.21	93.06
508	372	48	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	94.21	94.96	92.77	94.52	93.78	92.43	94.31	94.22	95.81	94.24	91.99	93.11	92.54	94.26	94.06	92.77	96.25	94.59	92.83	92.40	93.52	93.36	93.92	94.77	92.14
511	372	48	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	94.77	93.07	94.33	92.96	94.80	91.20	94.34	95.39	91.57	93.71	91.74	93.61	91.22	94.64	93.13	93.02	93.54	95.61	91.53	91.44	92.47	92.50	91.91	93.54	93.24
560	372	48	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	92.51	94.13	94.51	90.96	89.19	92.29	92.28	93.77	91.58	95.01	92.32	93.22	90.75	94.01	94.47	93.92	94.55	92.71	92.37	92.89	91.73	92.24	91.62	92.16	92.33
563	372	48	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	94.19	93.91	93.33	93.46	94.17	92.55	94.38	94.87	93.05	93.92	92.17	91.73	93.64	93.60	94.88	93.83	94.96	95.33	91.84	93.36	92.72	93.65	92.98	92.09	93.21
572	372	48	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	95.76	95.70	91.10	91.46	94.66	93.07	93.93	94.90	92.85	94.68	92.22	93.05	93.83	95.20	93.44	94.59	95.16	95.17	92.82	93.02	92.68	92.78	91.80	93.41	90.80
575	372	48	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	93.78	91.31	92.90	92.81	93.79	91.54	94.62	92.19	92.10	93.96	92.87	92.64	94.05	95.83	93.04	93.76	93.98	93.56	91.03	90.73	91.00	91.78	91.29	93.52	90.69
624	372	48	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	92.40	90.41	93.13	91.29	91.58	91.11	93.21	94.26	93.03	90.60	90.34	93.16	90.22	94.10	93.52	89.23	93.73	89.78	90.95	88.62	93.53	92.84	91.92	91.78	89.56
627	372	48	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	92.00	92.85	92.09	92.59	91.93	91.53	92.61	94.66	93.31	95.27	92.39	92.57	93.60	94.50	93.26	94.14	94.52	93.23	90.41	86.70	93.03	90.14	92.99	88.84	90.64
636	372	48	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	93.32	94.66	94.81	92.21	86.66	92.90	91.83	92.08	91.71	93.02	91.60	88.86	94.58	94.32	93.89	91.85	93.65	91.05	92.25	90.11	92.50	94.72	90.56	92.77	92.93
639	372	48	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	92.01	94.34	93.75	92.05	91.63	94.12	92.05	90.84	93.65	92.07	93.80	92.41	95.88	92.26	92.91	94.53	93.29	91.61	91.36	89.68	91.05	93.34	91.88	93.45	91.99
688	372	48	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	90.48	92.91	91.19	95.69	91.60	91.81	90.65	93.99	90.88	91.47	89.48	91.48	92.85	93.57	91.31	93.14	93.93	89.04	92.10	92.11	92.02	92.79	92.79	92.78	90.33
691	372	48	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	92.56	93.20	95.32	93.18	93.09	91.37	94.42	93.38	94.33	90.86	93.95	90.55	92.68	93.35	93.55	93.65	96.17	92.59	91.94	93.39	91.20	91.64	92.62	91.37	93.61
700	372	48	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	94.65	92.82	93.88	93.42	90.52	92.30	92.20	91.43	94.78	90.97	94.13	92.53	93.87	92.25	93.64	94.36	93.24	94.60	91.87	92.60	92.62	92.88	92.04	93.77	92.45
703	372	48	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	84.09	92.98	95.00	94.78	95.50	93.43	92.66	94.77	95.27	95.54	91.43	93.79	95.04	95.76	93.30	94.55	94.31	93.95	94.17	92.45	94.22	92.93	91.40	92.50	93.92
752	372	48	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	93.04	94.42	92.46	93.47	94.87	92.88	95.95	94.89	93.60	95.81	91.03	91.15	93.73	95.20	93.21	92.95	94.20	92.49	93.16	93.14	93.31	93.01	92.87	91.92	94.39
755	372	48	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	93.61	93.12	93.13	95.64	93.73	91.31	94.14	95.06	94.70	93.61	89.13	91.30	93.40	94.24	92.62	94.37	95.32	95.06	93.07	92.33	90.94	91.57	93.51	92.57	92.10
764	372	48	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	93.26	95.77	95.11	93.91	92.49	93.08	93.46	94.50	94.72	95.38	92.83	93.57	94.14	94.25	91.96	95.01	94.00	94.14	93.37	91.59	92.53	93.08	92.76	93.28	92.89
767	372	48	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	93.31	94.24	94.10	94.49	95.42	88.84	94.41	95.14	92.84	94.14	92.55	93.47	91.92	93.14	94.39	95.88	94.66	94.02	93.35	93.09	94.02	92.50	92.19	93.19	92.14
816	372	48	0.1	16	0.00010	36	0.000000e+00	0.000000e+00	93.68	92.46	94.48	91.29	94.69	90.99	92.91	94.12	93.03	92.57	91.04	79.73	91.10	93.28	91.59	93.58	94.32	90.48	91.86	88.52	94.10	91.86	92.19	91.94	91.57
819	372	48	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	90.98	92.18	91.20	94.71	94.92	90.16	91.70	94.12	92.75	93.69	90.72	94.85	92.31	94.52	92.10	94.30	93.68	95.44	89.61	92.20	93.16	91.58	93.79	92.56	89.25
828	372	48	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	94.34	94.73	94.60	93.13	94.63	92.93	93.91	95.32	95.14	95.57	92.46	91.28	92.41	93.72	93.94	94.94	93.50	92.56	92.98	91.51	93.08	92.99	93.38	92.28	92.84
831	372	48	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	94.08	94.99	95.21	95.90	95.36	92.22	92.31	93.23	94.90	94.55	91.00	92.95	94.25	93.76	92.74	94.58	94.80	94.86	92.73	92.36	93.63	95.76	94.34	92.95	93.90
880	372	48	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	92.40	92.87	92.94	93.26	93.87	91.27	92.12	92.96	92.15	94.46	91.88	92.58	94.24	93.19	92.86	92.07	92.54	92.91	94.31	90.11	92.69	90.63	92.52	91.51	90.71
883	372	48	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	95.11	48.12	93.09	92.39	94.22	92.91	92.95	92.19	91.92	94.78	92.61	93.52	94.75	95.46	94.73	95.69	94.58	93.89	90.92	89.78	93.04	48.12	93.12	91.87	92.18
892	372	48	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	93.34	91.69	92.76	95.43	94.73	93.48	92.11	89.66	92.88	94.73	90.15	90.52	91.22	94.28	94.79	93.85	94.33	91.94	92.14	91.84	90.14	93.08	92.70	92.15	93.39
895	372	48	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	94.16	95.02	92.13	91.97	94.28	92.94	92.81	93.17	94.60	92.61	90.26	91.75	91.50	93.80	92.38	92.85	94.41	93.53	92.08	93.42	90.67	93.61	92.82	91.86	91.81
944	372	48	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	94.75	94.02	93.38	95.63	95.22	94.28	94.91	94.80	95.06	94.58	92.54	93.23	94.19	95.02	91.99	94.44	95.10	92.67	91.99	90.35	93.50	94.93	93.42	92.32	93.28
947	372	48	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	93.92	93.92	94.02	93.23	93.09	93.38	92.60	92.54	91.06	93.19	91.61	93.58	93.99	94.58	95.14	93.54	95.46	93.42	92.59	92.70	81.10	91.48	92.97	93.00	93.73
956	372	48	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	93.70	93.56	92.03	94.89	93.04	91.31	93.31	91.71	94.03	92.18	91.26	92.40	93.41	94.68	92.92	94.27	96.04	92.30	94.01	92.87	91.64	95.19	92.57	92.07	93.06
959	372	48	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	93.13	95.89	95.50	95.80	94.37	93.62	96.40	94.32	94.55	95.30	93.44	93.04	95.66	95.37	93.27	95.37	95.69	92.68	94.15	93.37	93.35	93.55	92.48	93.19	92.79
1008	372	48	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	94.31	93.71	93.93	96.45	94.52	92.26	93.23	95.15	92.52	94.69	91.66	91.12	94.70	94.74	93.41	94.92	94.55	95.26	92.97	94.03	93.13	94.56	93.77	93.00	93.84
1011	372	48	0.1	16	0.00010	48	0.000000e+00	1.000000e-08	92.23	96.09	94.18	95.64	95.02	91.35	94.19	95.55	92.97	94.16	93.75	93.76	92.65	96.09	95.99	94.99	95.53	93.95	93.49	92.50	93.60	93.61	93.84	93.25	93.86
1020	372	48	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	92.41	95.67	93.92	94.52	93.21	92.80	95.35	92.28	92.95	95.16	91.69	91.60	92.16	93.56	93.44	93.44	93.55	93.04	93.45	92.31	90.62	92.69	93.94	92.55	92.10
1023	372	48	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	93.28	94.55	93.49	95.79	93.38	83.27	93.95	95.16	94.72	94.29	93.58	93.91	93.90	94.54	90.75	93.75	94.37	93.53	92.04	91.84	93.15	92.52	91.91	93.12	93.14
1072	372	48	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	92.52	94.85	94.43	95.55	94.73	91.69	91.07	91.07	93.06	92.99	92.47	94.02	94.79	94.47	93.53	94.48	92.80	93.95	92.43	92.97	93.26	94.93	92.54	92.34	93.20
1075	372	48	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	93.49	94.85	94.50	93.13	96.27	91.99	93.29	93.16	93.00	95.93	92.93	93.14	93.46	94.45	93.91	91.38	94.13	95.89	94.46	91.76	92.34	92.75	91.14	93.50	91.41
1084	372	48	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	93.87	92.20	93.45	94.87	94.46	92.63	91.70	92.63	94.57	95.12	91.70	91.67	94.30	94.98	92.07	95.26	96.38	92.74	92.54	91.48	92.70	93.12	91.53	92.21	93.46
1087	372	48	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	92.81	93.84	93.97	95.14	94.76	92.38	94.54	93.54	92.96	93.53	92.13	90.93	94.65	93.55	94.25	96.49	94.67	93.85	92.96	93.50	93.34	92.66	93.76	92.70	91.09
1136	372	48	0.1	64	0.00010	48	0.000000e+00	0.000000e+00	96.20	93.42	91.44	91.88	94.63	92.54	93.83	91.64	95.43	92.78	92.58	93.22	94.44	95.07	94.48	92.29	94.10	93.11	93.07	92.96	91.18	93.11	91.52	91.09	91.33
1139	372	48	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	94.38	94.97	94.38	95.28	94.40	93.33	93.33	92.46	95.31	94.36	93.53	93.02	95.01	93.02	94.24	94.17	94.55	94.15	91.36	92.03	93.28	92.03	93.56	93.56	93.21
1148	372	48	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	93.38	95.56	92.77	92.17	94.11	92.53	93.89	93.87	93.22	93.75	93.19	93.41	94.15	94.64	92.42	92.56	92.20	94.57	92.05	94.51	91.49	94.60	91.87	89.41	93.36
1151	372	48	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	93.24	93.56	96.29	93.81	95.42	93.19	94.56	94.74	93.71	93.80	93.11	90.28	96.01	94.81	93.84	95.88	93.81	94.27	91.99	92.35	90.87	92.99	94.25	92.70	92.39
1200	372	48	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	93.16	92.05	92.75	90.22	92.08	92.44	91.48	95.44	91.73	92.29	88.80	90.51	92.00	93.00	94.02	92.83	92.84	90.00	90.75	90.18	92.88	91.39	92.31	91.78	91.83
1203	372	48	0.1	16	0.00001	24	0.000000e+00	1.000000e-08	93.31	92.17	94.49	92.90	94.06	92.09	93.22	95.45	93.16	93.93	92.26	91.66	92.96	94.72	94.59	96.19	95.72	94.25	93.40	93.82	93.84	92.22	94.24	92.70	93.04
1212	372	48	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	92.68	93.71	93.03	93.52	94.51	92.76	94.51	92.39	94.12	95.03	93.60	92.59	94.09	96.34	90.96	92.87	92.25	95.81	92.20	90.62	92.29	94.49	92.91	92.11	92.63
1215	372	48	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	90.55	93.53	91.85	94.61	91.20	93.56	92.61	92.74	92.38	94.08	91.30	91.18	94.76	92.23	93.87	93.64	92.83	94.06	92.44	91.93	93.91	92.39	91.95	91.66	92.12
1264	372	48	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	91.43	92.57	93.60	92.88	93.57	93.07	92.33	92.56	93.26	95.39	94.09	90.26	94.98	94.53	92.91	94.09	93.06	93.23	91.81	89.70	92.44	90.72	92.77	91.06	89.37
1267	372	48	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	91.31	93.56	92.30	94.22	93.01	92.53	93.71	93.79	92.55	91.60	93.75	93.30	94.16	91.59	94.39	93.54	94.39	94.64	92.24	94.79	93.30	91.69	92.14	91.88	88.99
1276	372	48	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	93.66	92.50	93.88	93.63	92.56	94.59	94.42	92.46	93.22	94.52	92.42	92.43	93.17	94.56	93.38	94.50	96.05	94.91	93.22	93.98	91.43	92.14	93.71	92.22	91.17
1279	372	48	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	93.25	92.45	92.40	92.21	92.35	92.29	93.20	91.06	93.28	94.04	93.58	92.67	94.03	93.25	94.18	92.55	94.61	94.04	93.78	90.22	91.45	90.68	91.54	92.19	90.42
1328	372	48	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	93.19	95.10	94.26	89.79	93.22	94.08	90.66	92.42	94.06	94.68	93.32	91.78	94.59	96.77	92.83	92.76	95.03	95.58	93.15	91.98	91.20	93.87	94.27	92.05	91.46
1331	372	48	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	95.17	92.94	95.61	94.74	94.98	91.82	91.93	94.23	92.18	94.23	92.73	94.38	95.59	93.57	94.68	96.40	94.26	94.69	94.28	92.40	92.01	92.63	93.51	92.84	93.73
1340	372	48	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	94.28	94.38	94.37	94.62	93.37	92.37	94.79	92.56	94.39	94.78	93.40	94.49	93.88	95.06	92.24	95.56	94.73	93.54	94.35	93.91	92.21	91.85	94.38	93.24	92.63
1343	372	48	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	93.64	93.92	95.60	94.77	94.93	91.30	91.39	93.10	91.69	93.40	93.75	93.81	94.20	95.65	93.78	92.69	94.02	93.53	92.55	94.41	94.89	93.84	91.85	92.55	93.54
1392	372	48	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	93.02	91.84	94.23	91.08	93.50	91.87	93.00	95.09	93.32	92.71	93.26	91.45	93.54	94.65	93.74	93.77	93.70	92.59	90.84	90.68	93.60	92.80	94.01	92.24	92.24
1395	372	48	0.1	16	0.00001	36	0.000000e+00	1.000000e-08	94.31	94.29	95.22	96.07	95.29	92.95	93.16	95.27	92.93	95.93	93.50	85.47	94.60	95.32	95.88	94.71	96.42	95.55	92.83	92.48	93.21	95.33	93.25	94.08	91.63
1404	372	48	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	92.11	93.81	91.61	91.39	93.60	92.08	92.92	93.10	92.90	94.32	92.47	92.85	91.45	92.41	95.02	92.21	93.33	92.12	91.70	92.73	92.84	93.24	93.05	94.65	93.78
1407	372	48	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	91.38	94.04	94.88	95.96	94.28	91.68	93.86	90.60	95.12	93.92	93.05	93.35	95.41	93.75	91.32	94.23	92.65	92.84	94.00	91.96	91.79	90.35	93.01	94.60	92.87
1456	372	48	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	91.01	93.96	95.24	94.94	93.53	91.72	93.17	88.14	95.35	91.72	93.18	91.08	94.08	93.03	91.15	94.53	93.54	93.74	91.88	90.53	92.68	92.52	92.74	93.35	93.24
1459	372	48	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	94.46	94.62	93.29	93.30	92.62	92.66	91.63	93.41	92.13	94.27	91.57	92.20	93.96	93.82	91.34	95.55	95.11	92.01	93.01	91.86	92.76	92.68	93.27	92.89	91.48
1468	372	48	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	95.45	93.46	94.52	94.43	93.34	91.22	94.70	95.07	93.99	94.76	93.87	93.91	93.78	94.33	93.36	93.07	95.14	93.55	91.42	91.41	92.88	93.94	93.48	91.78	93.81
1471	372	48	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	93.04	94.32	95.51	91.97	95.26	92.57	94.78	93.55	93.33	95.08	92.89	91.94	91.30	95.10	92.88	94.06	93.07	94.96	93.84	90.36	93.06	94.26	92.97	92.60	91.87
1520	372	48	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	92.30	93.84	94.66	94.50	93.95	93.04	93.67	94.71	96.05	95.08	92.39	91.27	94.97	93.90	93.44	95.05	95.08	94.64	93.38	93.19	93.58	93.82	92.51	92.89	92.31
1523	372	48	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	94.25	94.80	95.10	95.30	95.90	93.08	95.21	93.49	95.06	95.90	93.02	92.22	95.28	95.56	94.87	95.62	94.41	93.68	91.70	93.52	92.77	91.96	92.99	91.74	93.52
1532	372	48	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	94.30	93.21	93.59	95.49	95.07	92.96	94.00	93.49	92.71	94.31	94.79	93.65	94.28	93.78	94.97	93.66	96.08	93.93	93.12	91.86	92.67	93.19	93.61	93.67	91.59
1535	372	48	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	94.87	94.80	94.27	95.36	94.47	93.55	92.80	95.32	93.43	93.66	93.12	92.23	93.32	92.50	94.79	95.24	93.06	93.55	94.58	92.18	94.48	92.58	93.41	91.74	93.62

Model performance on validation data (T-test)

Statistical Significance:

Using the global mean maximum row as the reference, we perform the right-tailed t-test to identify significant hyperparameters.

pvalues = []
pvalues_real = []

for _,index in enumerate(dt_valid_sub.index):
  _, p_value = stats.ttest_ind(max_row[:-1], dt_valid_sub.loc[index][:-1], alternative='greater')
  pvalues_real.append(p_value)
  pvalues.append((p_value >= 0.05)*1)

  

dt_valid_sub['pvalues'] = pvalues_real
dt_valid_sub['sig'] = pvalues


dt_valid_sub1 = dt_valid_sub.query('sig == 1')


max_indexx = dt_valid_sub1.iloc[:, :-3].mean().idxmax()

#print(sum(pvalues),max_indexx)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_abs_values	mAP_valid_abs_values_2	mAP_valid_abs_values_3	mAP_valid_abs_values_4	mAP_valid_abs_values_5	mAP_valid_abs_values_6	mAP_valid_abs_values_7	mAP_valid_abs_values_8	mAP_valid_abs_values_9	mAP_valid_abs_values_10	mAP_valid_abs_values_11	mAP_valid_abs_values_12	mAP_valid_abs_values_13	mAP_valid_abs_values_14	mAP_valid_abs_values_15	mAP_valid_abs_values_16	mAP_valid_abs_values_17	mAP_valid_abs_values_18	mAP_valid_abs_values_19	mAP_valid_abs_values_20	mAP_valid_abs_values_21	mAP_valid_abs_values_22	mAP_valid_abs_values_23	mAP_valid_abs_values_25	mAP_valid_abs_values_26
1020	372	48	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	85.60	87.07	86.35	90.41	85.34	87.96	85.23	87.80	86.79	89.02	88.58	87.93	86.76	85.46	86.77	87.08	87.41	87.66	88.73	89.30	89.08	89.63	88.91	89.66	89.09
1023	372	48	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	84.33	86.79	87.50	88.68	85.99	90.55	88.03	86.45	87.50	88.57	87.07	89.91	87.59	87.17	86.97	88.85	88.96	86.76	88.03	87.13	89.36	88.05	90.22	87.84	87.12
435	372	48	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	85.31	87.36	87.02	85.20	88.81	86.82	87.76	85.98	88.76	88.09	91.31	88.19	88.26	85.73	87.49	88.29	86.44	88.29	90.18	87.33	90.03	89.41	89.05	87.39	88.70
1468	372	48	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	87.90	86.66	86.71	85.40	87.97	89.21	86.53	84.85	88.66	86.64	88.41	87.14	87.23	87.33	87.33	88.78	88.90	86.67	91.27	87.59	89.03	88.67	89.27	88.59	89.84
1471	372	48	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	86.25	86.75	87.45	90.66	84.77	86.47	88.10	86.77	84.65	86.68	88.72	89.57	89.73	83.16	86.04	88.15	85.51	89.01	88.81	91.07	89.49	89.02	90.18	88.63	87.06
691	372	48	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	87.37	86.62	87.45	86.35	88.49	89.31	86.41	90.66	87.17	85.77	89.63	88.47	87.05	88.37	87.99	87.58	86.36	86.24	90.28	89.40	88.24	93.05	89.50	90.33	89.35
892	372	48	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	87.05	87.43	87.24	85.47	86.21	88.10	86.70	88.84	88.14	87.37	89.02	87.30	88.20	86.63	86.45	88.11	86.91	86.36	88.77	89.88	88.52	88.89	89.66	88.68	89.26
895	372	48	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	85.61	88.38	85.63	87.49	87.86	87.52	85.12	86.15	86.89	87.75	88.74	87.45	89.02	87.14	87.37	88.88	86.57	87.43	88.44	89.52	89.36	89.52	89.02	90.31	88.79
1084	372	48	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	88.63	88.33	87.15	86.58	89.23	88.69	88.24	87.40	89.00	86.43	87.32	89.68	87.52	88.24	88.39	89.32	85.88	87.85	88.06	89.80	89.57	88.35	87.96	89.94	90.95
127	372	48	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	84.52	85.82	86.31	86.99	87.74	87.99	86.04	86.08	88.75	87.09	87.63	90.45	88.72	87.64	90.28	87.88	87.44	86.95	88.64	88.30	88.01	89.05	88.02	88.49	88.94
307	372	48	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	87.32	88.00	86.61	86.65	86.67	89.92	88.18	87.08	89.73	86.18	87.98	89.14	87.43	87.64	87.01	89.51	86.58	88.45	89.12	89.51	89.09	87.84	88.45	89.98	86.23
316	372	48	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	87.97	87.43	86.48	86.58	87.77	86.80	85.76	85.76	87.22	88.65	87.76	87.33	88.35	87.87	87.71	86.89	86.82	87.85	92.34	89.92	89.74	89.21	89.32	91.17	91.82
319	372	48	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	86.55	86.72	86.63	87.15	86.06	88.33	89.32	86.37	88.37	86.80	89.27	88.61	87.47	87.79	86.53	85.97	87.84	85.55	88.32	89.40	88.20	90.20	88.40	89.82	88.98
508	372	48	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	88.36	86.85	87.99	87.14	86.38	87.14	87.33	86.59	86.02	86.24	90.63	89.15	89.56	86.25	86.79	87.56	85.76	86.38	88.67	89.39	89.22	88.00	87.90	88.63	90.68
1535	372	48	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	85.43	87.50	89.00	86.80	86.12	86.31	86.95	85.54	86.01	86.10	87.92	88.16	87.81	87.67	86.52	87.75	90.14	87.07	89.75	92.23	88.93	88.88	88.37	88.85	88.22
755	372	48	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	87.15	89.30	86.12	86.29	88.10	87.95	87.62	89.40	88.18	85.85	89.19	89.72	89.27	87.64	85.49	85.87	87.27	87.85	89.46	89.46	90.70	89.08	87.53	89.91	90.16
188	372	48	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	84.51	88.86	86.31	85.30	87.49	88.14	87.27	88.62	87.45	88.43	88.58	87.60	87.37	83.51	84.52	89.82	85.67	87.12	88.59	90.26	89.54	88.72	90.84	88.24	88.59

Size of the All data:  (96, 28)
Size of the Sig data:  (17, 28)

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params1 = dt_valid.loc[dt_valid_sub1.index][['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
optimal_params = optimal_params1.reset_index()
optimal_params.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_1 = pd.merge(optimal_params, dt_test, on=['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg'])
test_data_1_sub = test_data_1[list(test_data_1.columns[test_data_1.columns.str.startswith('mAP_test')])]

Equivalent test data

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_abs_values	mAP_test_abs_values_2	mAP_test_abs_values_3	mAP_test_abs_values_4	mAP_test_abs_values_5	mAP_test_abs_values_6	mAP_test_abs_values_7	mAP_test_abs_values_8	mAP_test_abs_values_9	mAP_test_abs_values_10	mAP_test_abs_values_11	mAP_test_abs_values_12	mAP_test_abs_values_13	mAP_test_abs_values_14	mAP_test_abs_values_15	mAP_test_abs_values_16	mAP_test_abs_values_17	mAP_test_abs_values_18	mAP_test_abs_values_19	mAP_test_abs_values_20	mAP_test_abs_values_21	mAP_test_abs_values_22	mAP_test_abs_values_23	mAP_test_abs_values_25	mAP_test_abs_values_26
14	372	48	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	92.41	95.67	93.92	94.52	93.21	92.80	95.35	92.28	92.95	95.16	91.69	91.60	92.16	93.56	93.44	93.44	93.55	93.04	93.45	92.31	90.62	92.69	93.94	92.55	92.10
8	372	48	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	93.28	94.55	93.49	95.79	93.38	83.27	93.95	95.16	94.72	94.29	93.58	93.91	93.90	94.54	90.75	93.75	94.37	93.53	92.04	91.84	93.15	92.52	91.91	93.12	93.14
5	372	48	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	93.23	95.25	92.71	93.39	93.96	91.78	92.47	93.87	94.74	94.94	91.90	94.74	93.30	92.93	94.05	95.55	94.67	93.63	92.65	89.88	94.25	92.06	92.78	91.43	93.27
6	372	48	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	95.45	93.46	94.52	94.43	93.34	91.22	94.70	95.07	93.99	94.76	93.87	93.91	93.78	94.33	93.36	93.07	95.14	93.55	91.42	91.41	92.88	93.94	93.48	91.78	93.81
15	372	48	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	93.04	94.32	95.51	91.97	95.26	92.57	94.78	93.55	93.33	95.08	92.89	91.94	91.30	95.10	92.88	94.06	93.07	94.96	93.84	90.36	93.06	94.26	92.97	92.60	91.87
1	372	48	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	92.56	93.20	95.32	93.18	93.09	91.37	94.42	93.38	94.33	90.86	93.95	90.55	92.68	93.35	93.55	93.65	96.17	92.59	91.94	93.39	91.20	91.64	92.62	91.37	93.61
9	372	48	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	93.34	91.69	92.76	95.43	94.73	93.48	92.11	89.66	92.88	94.73	90.15	90.52	91.22	94.28	94.79	93.85	94.33	91.94	92.14	91.84	90.14	93.08	92.70	92.15	93.39
7	372	48	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	94.16	95.02	92.13	91.97	94.28	92.94	92.81	93.17	94.60	92.61	90.26	91.75	91.50	93.80	92.38	92.85	94.41	93.53	92.08	93.42	90.67	93.61	92.82	91.86	91.81
0	372	48	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	93.87	92.20	93.45	94.87	94.46	92.63	91.70	92.63	94.57	95.12	91.70	91.67	94.30	94.98	92.07	95.26	96.38	92.74	92.54	91.48	92.70	93.12	91.53	92.21	93.46
13	372	48	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	92.52	92.37	94.59	93.73	95.07	92.57	93.22	92.69	95.96	93.10	94.14	90.79	94.15	96.09	89.53	93.70	94.96	93.76	94.41	92.78	92.75	92.36	89.65	91.54	89.07
4	372	48	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	93.71	94.56	90.96	94.57	92.55	89.98	95.28	94.57	94.32	93.75	93.04	90.96	95.36	93.09	92.27	91.81	95.07	94.65	90.15	92.35	93.34	91.22	93.76	91.11	91.68
3	372	48	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	93.60	95.85	93.82	95.20	93.83	91.42	96.37	93.65	94.94	93.29	92.14	94.10	92.97	94.34	92.59	93.16	95.08	92.05	92.36	91.99	93.15	92.27	92.37	93.10	92.66
10	372	48	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	93.95	91.41	90.78	95.00	94.22	91.58	94.07	94.24	93.00	93.35	92.59	93.98	94.57	94.77	94.41	95.55	91.64	91.90	92.53	93.35	93.70	93.71	92.50	92.18	91.75
11	372	48	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	94.21	94.96	92.77	94.52	93.78	92.43	94.31	94.22	95.81	94.24	91.99	93.11	92.54	94.26	94.06	92.77	96.25	94.59	92.83	92.40	93.52	93.36	93.92	94.77	92.14
12	372	48	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	94.87	94.80	94.27	95.36	94.47	93.55	92.80	95.32	93.43	93.66	93.12	92.23	93.32	92.50	94.79	95.24	93.06	93.55	94.58	92.18	94.48	92.58	93.41	91.74	93.62
2	372	48	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	93.61	93.12	93.13	95.64	93.73	91.31	94.14	95.06	94.70	93.61	89.13	91.30	93.40	94.24	92.62	94.37	95.32	95.06	93.07	92.33	90.94	91.57	93.51	92.57	92.10
16	372	48	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	93.94	92.61	92.99	91.60	93.62	92.79	94.98	94.40	93.57	94.08	91.30	89.75	94.38	91.24	94.74	93.57	95.51	94.78	93.46	89.55	93.61	91.50	94.72	92.09	88.73

Size of the test data:  (17, 33)

Difference

diff = dt[['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
for i, cols in enumerate(zip(dt.columns,test_data_1.columns)):   
   diff[f'col_diff_{i}'] = (np.array(dt[cols[0]]) - np.array(test_data_1[cols[1]]))*-1

# Function to count negatives in a row
def count_negatives(row):
    return sum(1 for value in row if isinstance(value, (int, float)) and value < 0)

diff['negative_count'] = diff.apply(count_negatives, axis=1)
diff.sort_values(by =["negative_count"],inplace = True, ascending=False)

html_table = diff.to_html(index=True)

# Wrap in a scrollable div
scrollable_table = f"""
<div style="height: 400px; width: 100%; overflow-x: auto; overflow-y: auto;">
    {html_table}
</div>
"""
# Display the scrollable table
display(HTML(scrollable_table))

# 
# show(diff)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	col_diff_0	col_diff_1	col_diff_2	col_diff_3	col_diff_4	col_diff_5	col_diff_6	col_diff_7	col_diff_8	col_diff_9	col_diff_10	col_diff_11	col_diff_12	col_diff_13	col_diff_14	col_diff_15	col_diff_16	col_diff_17	col_diff_18	col_diff_19	col_diff_20	col_diff_21	col_diff_22	col_diff_23	col_diff_24	col_diff_25	col_diff_26	col_diff_27	col_diff_28	col_diff_29	col_diff_30	col_diff_31	col_diff_32	negative_count
188	372	48	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.43	3.75	6.68	6.30	6.13	4.65	7.71	5.78	6.12	5.65	2.72	2.15	7.01	7.73	10.22	3.75	9.84	7.66	4.87	-0.71	4.07	2.78	3.88	3.85	0.14	1
755	372	48	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.46	3.82	7.01	9.35	5.63	3.36	6.52	5.66	6.52	7.76	-0.06	1.58	4.13	6.60	7.13	8.50	8.05	7.21	3.61	2.87	0.24	2.49	5.98	2.66	1.94	1
1535	372	48	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.44	7.30	5.27	8.56	8.35	7.24	5.85	9.78	7.42	7.56	5.20	4.07	5.51	4.83	8.27	7.49	2.92	6.48	4.83	-0.05	5.55	3.70	5.04	2.89	5.40	1
1471	372	48	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.79	7.57	8.06	1.31	10.49	6.10	6.68	6.78	8.68	8.40	4.17	2.37	1.57	11.94	6.84	5.91	7.56	5.95	5.03	-0.71	3.57	5.24	2.79	3.97	4.81	1
691	372	48	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.19	6.58	7.87	6.83	4.60	2.06	8.01	2.72	7.16	5.09	4.32	2.08	5.63	4.98	5.56	6.07	9.81	6.35	1.66	3.99	2.96	-1.41	3.12	1.04	4.26	1
1023	372	48	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.95	7.76	5.99	7.11	7.39	-7.28	5.92	8.71	7.22	5.72	6.51	4.00	6.31	7.37	3.78	4.90	5.41	6.77	4.01	4.71	3.79	4.47	1.69	5.28	6.02	1
127	372	48	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.00	6.55	8.28	6.74	7.33	4.58	7.18	6.61	7.21	6.01	6.51	0.34	5.43	8.45	-0.75	5.82	7.52	6.81	5.77	4.48	4.74	3.31	1.63	3.05	0.13	1
316	372	48	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.63	8.42	7.34	8.62	6.06	4.62	10.61	7.89	7.72	4.64	4.38	6.77	4.62	6.47	4.88	6.27	8.26	4.20	0.02	2.07	3.41	3.06	3.05	1.93	0.84	0
508	372	48	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.85	8.11	4.78	7.38	7.40	5.29	6.98	7.63	9.79	8.00	1.36	3.96	2.98	8.01	7.27	5.21	10.49	8.21	4.16	3.01	4.30	5.36	6.02	6.14	1.46	0
319	372	48	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.40	4.69	4.15	7.85	8.16	3.25	4.75	7.87	4.63	6.55	3.32	5.37	7.10	6.98	7.88	9.58	3.80	6.35	4.21	3.95	5.50	3.51	4.10	2.36	2.77	0
1020	372	48	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.81	8.60	7.57	4.11	7.87	4.84	10.12	4.48	6.16	6.14	3.11	3.67	5.40	8.10	6.67	6.36	6.14	5.38	4.72	3.01	1.54	3.06	5.03	2.89	3.01	0
307	372	48	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.39	6.56	4.35	7.92	5.88	0.06	7.10	7.49	4.59	7.57	5.06	1.82	7.93	5.45	5.26	2.30	8.49	6.20	1.03	2.84	4.25	3.38	5.31	1.13	5.45	0
895	372	48	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.55	6.64	6.50	4.48	6.42	5.42	7.69	7.02	7.71	4.86	1.52	4.30	2.48	6.66	5.01	3.97	7.84	6.10	3.64	3.90	1.31	4.09	3.80	1.55	3.02	0
892	372	48	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.29	4.26	5.52	9.96	8.52	5.38	5.41	0.82	4.74	7.36	1.13	3.22	3.02	7.65	8.34	5.74	7.42	5.58	3.37	1.96	1.62	4.19	3.04	3.47	4.13	0
1468	372	48	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.55	6.80	7.81	9.03	5.37	2.01	8.17	10.22	5.33	8.12	5.46	6.77	6.55	7.00	6.03	4.29	6.24	6.88	0.15	3.82	3.85	5.27	4.21	3.19	3.97	0
435	372	48	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.92	7.89	5.69	8.19	5.15	4.96	4.71	7.89	5.98	6.85	0.59	6.55	5.04	7.20	6.56	7.26	8.23	5.34	2.47	2.55	4.22	2.65	3.73	4.04	4.57	0
1084	372	48	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.24	3.87	6.30	8.29	5.23	3.94	3.46	5.23	5.57	8.69	4.38	1.99	6.78	6.74	3.68	5.94	10.50	4.89	4.48	1.68	3.13	4.77	3.57	2.27	2.51	0

Model performance on valid data

Average & Std Deviation of the Significant rows:

	mean	std
mAP_valid_abs_values_20	89.381765	1.278589
mAP_valid_abs_values_19	89.262353	1.175976
mAP_valid_abs_values_25	89.203529	1.018945
mAP_valid_abs_values_21	89.182941	0.686575
mAP_valid_abs_values_22	89.151176	1.178643
mAP_valid_abs_values_26	89.045882	1.429082
mAP_valid_abs_values_23	88.976471	0.921995
mAP_valid_abs_values_11	88.691765	1.115226
mAP_valid_abs_values_12	88.576471	1.050880
mAP_valid_abs_values_13	88.078824	0.908053
mAP_valid_abs_values_6	88.071176	1.196379
mAP_valid_abs_values_16	88.017059	1.138452
mAP_valid_abs_values_9	87.605294	1.293204
mAP_valid_abs_values_2	87.404118	0.910385
mAP_valid_abs_values_18	87.264118	0.903853
mAP_valid_abs_values_10	87.156471	1.069006
mAP_valid_abs_values_5	87.117647	1.275463
mAP_valid_abs_values_7	87.093529	1.155595
mAP_valid_abs_values_17	87.085882	1.280430
mAP_valid_abs_values_8	87.078824	1.534759
mAP_valid_abs_values_15	87.038235	1.245985
mAP_valid_abs_values_4	87.008235	1.592954
mAP_valid_abs_values_3	86.938235	0.803129
mAP_valid_abs_values_14	86.778824	1.529064
mAP_valid_abs_values	86.462353	1.393402

Model performance on Test data

We then apply the hyperparameters to the test set and average the results.

test_data_1['average_map'] = test_data_1_sub.apply(np.mean, axis=1)

test_data_1.sort_values(by=['average_map'], ascending= False,inplace=True)

	mean	std
mAP_test_abs_values_17	94.645882	1.257040
mAP_test_abs_values_9	94.225882	0.932128
mAP_test_abs_values_4	94.186471	1.336128
mAP_test_abs_values_7	93.968235	1.276872
mAP_test_abs_values_14	93.964706	1.127126
mAP_test_abs_values_5	93.940000	0.726051
mAP_test_abs_values_10	93.919412	1.108397
mAP_test_abs_values_16	93.861765	1.053086
mAP_test_abs_values_2	93.825882	1.417705
mAP_test_abs_values_8	93.701176	1.397130
mAP_test_abs_values	93.632353	0.802773
mAP_test_abs_values_18	93.520588	1.046174
mAP_test_abs_values_3	93.360000	1.321708
mAP_test_abs_values_13	93.225294	1.203854
average_map	93.211318	0.306089
mAP_test_abs_values_15	93.075294	1.444641
mAP_test_abs_values_23	92.858235	1.148843
mAP_test_abs_values_19	92.675882	1.091639
mAP_test_abs_values_22	92.675882	0.913572
mAP_test_abs_values_21	92.597647	1.351409
mAP_test_abs_values_26	92.247647	1.466226
mAP_test_abs_values_25	92.245294	0.868548
mAP_test_abs_values_11	92.202353	1.426278
mAP_test_abs_values_12	92.165294	1.514117
mAP_test_abs_values_20	91.932941	1.136858
mAP_test_abs_values_6	91.628824	2.344754

Model performance on validation data (Mann-Whitney U Test)

Statistical Significance:

Using the global mean maximum row as the reference, we perform the right-tailed t-test to identify significant hyperparameters.

pvalues_mw = []
pvalues_real_mw = []

for _,index in enumerate(dt_valid_sub.index):

  _, p_value_mw = stats.mannwhitneyu(max_row[:-1], dt_valid_sub.loc[index][:-1], alternative='greater')#, method = 'asymptotic')
  pvalues_real_mw.append(p_value_mw)
  pvalues_mw.append((p_value_mw >= 0.05)*1)
  

dt_valid_sub_mw['pvalues_mw'] = pvalues_real_mw
dt_valid_sub_mw['sig_mw'] = pvalues_mw

dt_valid_sub_mw1 = dt_valid_sub_mw.query('sig_mw == 1')


max_indexx = dt_valid_sub_mw1.iloc[:, :-3].mean().idxmax()

#print(sum(pvalues),max_indexx)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_abs_values	mAP_valid_abs_values_2	mAP_valid_abs_values_3	mAP_valid_abs_values_4	mAP_valid_abs_values_5	mAP_valid_abs_values_6	mAP_valid_abs_values_7	mAP_valid_abs_values_8	mAP_valid_abs_values_9	mAP_valid_abs_values_10	mAP_valid_abs_values_11	mAP_valid_abs_values_12	mAP_valid_abs_values_13	mAP_valid_abs_values_14	mAP_valid_abs_values_15	mAP_valid_abs_values_16	mAP_valid_abs_values_17	mAP_valid_abs_values_18	mAP_valid_abs_values_19	mAP_valid_abs_values_20	mAP_valid_abs_values_21	mAP_valid_abs_values_22	mAP_valid_abs_values_23	mAP_valid_abs_values_25	mAP_valid_abs_values_26
1020	372	48	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	85.60	87.07	86.35	90.41	85.34	87.96	85.23	87.80	86.79	89.02	88.58	87.93	86.76	85.46	86.77	87.08	87.41	87.66	88.73	89.30	89.08	89.63	88.91	89.66	89.09
435	372	48	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	85.31	87.36	87.02	85.20	88.81	86.82	87.76	85.98	88.76	88.09	91.31	88.19	88.26	85.73	87.49	88.29	86.44	88.29	90.18	87.33	90.03	89.41	89.05	87.39	88.70
1468	372	48	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	87.90	86.66	86.71	85.40	87.97	89.21	86.53	84.85	88.66	86.64	88.41	87.14	87.23	87.33	87.33	88.78	88.90	86.67	91.27	87.59	89.03	88.67	89.27	88.59	89.84
1471	372	48	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	86.25	86.75	87.45	90.66	84.77	86.47	88.10	86.77	84.65	86.68	88.72	89.57	89.73	83.16	86.04	88.15	85.51	89.01	88.81	91.07	89.49	89.02	90.18	88.63	87.06
691	372	48	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	87.37	86.62	87.45	86.35	88.49	89.31	86.41	90.66	87.17	85.77	89.63	88.47	87.05	88.37	87.99	87.58	86.36	86.24	90.28	89.40	88.24	93.05	89.50	90.33	89.35
895	372	48	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	85.61	88.38	85.63	87.49	87.86	87.52	85.12	86.15	86.89	87.75	88.74	87.45	89.02	87.14	87.37	88.88	86.57	87.43	88.44	89.52	89.36	89.52	89.02	90.31	88.79
1084	372	48	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	88.63	88.33	87.15	86.58	89.23	88.69	88.24	87.40	89.00	86.43	87.32	89.68	87.52	88.24	88.39	89.32	85.88	87.85	88.06	89.80	89.57	88.35	87.96	89.94	90.95
307	372	48	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	87.32	88.00	86.61	86.65	86.67	89.92	88.18	87.08	89.73	86.18	87.98	89.14	87.43	87.64	87.01	89.51	86.58	88.45	89.12	89.51	89.09	87.84	88.45	89.98	86.23
316	372	48	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	87.97	87.43	86.48	86.58	87.77	86.80	85.76	85.76	87.22	88.65	87.76	87.33	88.35	87.87	87.71	86.89	86.82	87.85	92.34	89.92	89.74	89.21	89.32	91.17	91.82
319	372	48	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	86.55	86.72	86.63	87.15	86.06	88.33	89.32	86.37	88.37	86.80	89.27	88.61	87.47	87.79	86.53	85.97	87.84	85.55	88.32	89.40	88.20	90.20	88.40	89.82	88.98
755	372	48	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	87.15	89.30	86.12	86.29	88.10	87.95	87.62	89.40	88.18	85.85	89.19	89.72	89.27	87.64	85.49	85.87	87.27	87.85	89.46	89.46	90.70	89.08	87.53	89.91	90.16
188	372	48	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	84.51	88.86	86.31	85.30	87.49	88.14	87.27	88.62	87.45	88.43	88.58	87.60	87.37	83.51	84.52	89.82	85.67	87.12	88.59	90.26	89.54	88.72	90.84	88.24	88.59

Size of the All data:  (96, 28)
Size of the Sig data:  (12, 33)

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params_mw1 = dt_valid.loc[dt_valid_sub_mw1.index][['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
optimal_params_mw = optimal_params_mw1.reset_index()
optimal_params_mw.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_mw1 = pd.merge(optimal_params_mw, dt_test, on=['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg'])
test_data_1_sub_mw = test_data_mw1[list(test_data_mw1.columns[test_data_mw1.columns.str.startswith('mAP_test')])]

Equivalent test data

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_abs_values	mAP_test_abs_values_2	mAP_test_abs_values_3	mAP_test_abs_values_4	mAP_test_abs_values_5	mAP_test_abs_values_6	mAP_test_abs_values_7	mAP_test_abs_values_8	mAP_test_abs_values_9	mAP_test_abs_values_10	mAP_test_abs_values_11	mAP_test_abs_values_12	mAP_test_abs_values_13	mAP_test_abs_values_14	mAP_test_abs_values_15	mAP_test_abs_values_16	mAP_test_abs_values_17	mAP_test_abs_values_18	mAP_test_abs_values_19	mAP_test_abs_values_20	mAP_test_abs_values_21	mAP_test_abs_values_22	mAP_test_abs_values_23	mAP_test_abs_values_25	mAP_test_abs_values_26
9	372	48	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	92.41	95.67	93.92	94.52	93.21	92.80	95.35	92.28	92.95	95.16	91.69	91.60	92.16	93.56	93.44	93.44	93.55	93.04	93.45	92.31	90.62	92.69	93.94	92.55	92.10
5	372	48	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	93.23	95.25	92.71	93.39	93.96	91.78	92.47	93.87	94.74	94.94	91.90	94.74	93.30	92.93	94.05	95.55	94.67	93.63	92.65	89.88	94.25	92.06	92.78	91.43	93.27
6	372	48	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	95.45	93.46	94.52	94.43	93.34	91.22	94.70	95.07	93.99	94.76	93.87	93.91	93.78	94.33	93.36	93.07	95.14	93.55	91.42	91.41	92.88	93.94	93.48	91.78	93.81
10	372	48	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	93.04	94.32	95.51	91.97	95.26	92.57	94.78	93.55	93.33	95.08	92.89	91.94	91.30	95.10	92.88	94.06	93.07	94.96	93.84	90.36	93.06	94.26	92.97	92.60	91.87
1	372	48	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	92.56	93.20	95.32	93.18	93.09	91.37	94.42	93.38	94.33	90.86	93.95	90.55	92.68	93.35	93.55	93.65	96.17	92.59	91.94	93.39	91.20	91.64	92.62	91.37	93.61
7	372	48	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	94.16	95.02	92.13	91.97	94.28	92.94	92.81	93.17	94.60	92.61	90.26	91.75	91.50	93.80	92.38	92.85	94.41	93.53	92.08	93.42	90.67	93.61	92.82	91.86	91.81
0	372	48	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	93.87	92.20	93.45	94.87	94.46	92.63	91.70	92.63	94.57	95.12	91.70	91.67	94.30	94.98	92.07	95.26	96.38	92.74	92.54	91.48	92.70	93.12	91.53	92.21	93.46
4	372	48	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	93.71	94.56	90.96	94.57	92.55	89.98	95.28	94.57	94.32	93.75	93.04	90.96	95.36	93.09	92.27	91.81	95.07	94.65	90.15	92.35	93.34	91.22	93.76	91.11	91.68
3	372	48	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	93.60	95.85	93.82	95.20	93.83	91.42	96.37	93.65	94.94	93.29	92.14	94.10	92.97	94.34	92.59	93.16	95.08	92.05	92.36	91.99	93.15	92.27	92.37	93.10	92.66
8	372	48	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	93.95	91.41	90.78	95.00	94.22	91.58	94.07	94.24	93.00	93.35	92.59	93.98	94.57	94.77	94.41	95.55	91.64	91.90	92.53	93.35	93.70	93.71	92.50	92.18	91.75
2	372	48	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	93.61	93.12	93.13	95.64	93.73	91.31	94.14	95.06	94.70	93.61	89.13	91.30	93.40	94.24	92.62	94.37	95.32	95.06	93.07	92.33	90.94	91.57	93.51	92.57	92.10
11	372	48	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	93.94	92.61	92.99	91.60	93.62	92.79	94.98	94.40	93.57	94.08	91.30	89.75	94.38	91.24	94.74	93.57	95.51	94.78	93.46	89.55	93.61	91.50	94.72	92.09	88.73

Size of the test data:  (12, 33)

Difference

diff_mw = dt_mw[['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
for i, cols in enumerate(zip(dt_mw.columns,test_data_mw1.columns)):   
   diff_mw[f'col_diff_{i}'] = (np.array(dt_mw[cols[0]]) - np.array(test_data_mw1[cols[1]]))*-1

# Function to count negatives in a row
def count_negatives(row):
    return sum(1 for value in row if isinstance(value, (int, float)) and value < 0)

diff_mw['negative_count'] = diff_mw.apply(count_negatives, axis=1)

diff_mw.sort_values(by =["negative_count"],inplace = True, ascending=False)


html_table = diff_mw.to_html(index=True)

# Wrap in a scrollable div
scrollable_table = f"""
<div style="height: 400px; width: 100%; overflow-x: auto; overflow-y: auto;">
    {html_table}
</div>
"""
# Display the scrollable table
display(HTML(scrollable_table))

key_values = ['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']
print('Are the keys of the valid and test dfs same?: ',dt_mw[key_values].equals(test_data_mw1[key_values]))

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	col_diff_0	col_diff_1	col_diff_2	col_diff_3	col_diff_4	col_diff_5	col_diff_6	col_diff_7	col_diff_8	col_diff_9	col_diff_10	col_diff_11	col_diff_12	col_diff_13	col_diff_14	col_diff_15	col_diff_16	col_diff_17	col_diff_18	col_diff_19	col_diff_20	col_diff_21	col_diff_22	col_diff_23	col_diff_24	col_diff_25	col_diff_26	col_diff_27	col_diff_28	col_diff_29	col_diff_30	col_diff_31	col_diff_32	negative_count
1471	372	48	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.79	7.57	8.06	1.31	10.49	6.10	6.68	6.78	8.68	8.40	4.17	2.37	1.57	11.94	6.84	5.91	7.56	5.95	5.03	-0.71	3.57	5.24	2.79	3.97	4.81	1
691	372	48	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.19	6.58	7.87	6.83	4.60	2.06	8.01	2.72	7.16	5.09	4.32	2.08	5.63	4.98	5.56	6.07	9.81	6.35	1.66	3.99	2.96	-1.41	3.12	1.04	4.26	1
755	372	48	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.46	3.82	7.01	9.35	5.63	3.36	6.52	5.66	6.52	7.76	-0.06	1.58	4.13	6.60	7.13	8.50	8.05	7.21	3.61	2.87	0.24	2.49	5.98	2.66	1.94	1
188	372	48	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.43	3.75	6.68	6.30	6.13	4.65	7.71	5.78	6.12	5.65	2.72	2.15	7.01	7.73	10.22	3.75	9.84	7.66	4.87	-0.71	4.07	2.78	3.88	3.85	0.14	1
1020	372	48	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.81	8.60	7.57	4.11	7.87	4.84	10.12	4.48	6.16	6.14	3.11	3.67	5.40	8.10	6.67	6.36	6.14	5.38	4.72	3.01	1.54	3.06	5.03	2.89	3.01	0
435	372	48	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.92	7.89	5.69	8.19	5.15	4.96	4.71	7.89	5.98	6.85	0.59	6.55	5.04	7.20	6.56	7.26	8.23	5.34	2.47	2.55	4.22	2.65	3.73	4.04	4.57	0
1468	372	48	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.55	6.80	7.81	9.03	5.37	2.01	8.17	10.22	5.33	8.12	5.46	6.77	6.55	7.00	6.03	4.29	6.24	6.88	0.15	3.82	3.85	5.27	4.21	3.19	3.97	0
895	372	48	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.55	6.64	6.50	4.48	6.42	5.42	7.69	7.02	7.71	4.86	1.52	4.30	2.48	6.66	5.01	3.97	7.84	6.10	3.64	3.90	1.31	4.09	3.80	1.55	3.02	0
1084	372	48	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.24	3.87	6.30	8.29	5.23	3.94	3.46	5.23	5.57	8.69	4.38	1.99	6.78	6.74	3.68	5.94	10.50	4.89	4.48	1.68	3.13	4.77	3.57	2.27	2.51	0
307	372	48	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.39	6.56	4.35	7.92	5.88	0.06	7.10	7.49	4.59	7.57	5.06	1.82	7.93	5.45	5.26	2.30	8.49	6.20	1.03	2.84	4.25	3.38	5.31	1.13	5.45	0
316	372	48	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.63	8.42	7.34	8.62	6.06	4.62	10.61	7.89	7.72	4.64	4.38	6.77	4.62	6.47	4.88	6.27	8.26	4.20	0.02	2.07	3.41	3.06	3.05	1.93	0.84	0
319	372	48	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.40	4.69	4.15	7.85	8.16	3.25	4.75	7.87	4.63	6.55	3.32	5.37	7.10	6.98	7.88	9.58	3.80	6.35	4.21	3.95	5.50	3.51	4.10	2.36	2.77	0

Are the keys of the valid and test dfs same?:  False

Model performance on valid data

Average & Std Deviation of the Significant rows:

	mean	std
mAP_valid_abs_values_25	89.497500	1.065092
mAP_valid_abs_values_19	89.466667	1.310401
mAP_valid_abs_values_22	89.391667	1.308829
mAP_valid_abs_values_20	89.380000	1.024926
mAP_valid_abs_values_21	89.339167	0.696569
mAP_valid_abs_values_26	89.130000	1.521751
mAP_valid_abs_values_23	89.035833	0.909740
mAP_valid_abs_values_11	88.790833	1.025329
mAP_valid_abs_values_12	88.402500	0.948023
mAP_valid_abs_values_6	88.093333	1.077643
mAP_valid_abs_values_16	88.011667	1.339449
mAP_valid_abs_values_13	87.955000	0.957179
mAP_valid_abs_values_9	87.739167	1.344264
mAP_valid_abs_values_2	87.623333	0.928374
mAP_valid_abs_values_18	87.497500	0.973654
mAP_valid_abs_values_5	87.380000	1.387634
mAP_valid_abs_values_8	87.236667	1.659328
mAP_valid_abs_values_10	87.190833	1.138823
mAP_valid_abs_values_7	87.128333	1.318869
mAP_valid_abs_values_4	87.005000	1.793369
mAP_valid_abs_values_15	86.886667	1.102644
mAP_valid_abs_values_17	86.770833	0.967033
mAP_valid_abs_values	86.680833	1.249017
mAP_valid_abs_values_3	86.659167	0.541655
mAP_valid_abs_values_14	86.656667	1.789842

Model performance on Test data

We then apply the hyperparameters to the test set and average the results.

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params = dt_valid.loc[dt_valid_sub_mw1.index].iloc[:, :6].reset_index()
optimal_params.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_1 = pd.merge(optimal_params, dt_test, how='left')
test_data_1_sub = test_data_1[list(test_data_1.columns[test_data_1.columns.str.startswith('mAP_test')])]
test_data_1['average_map'] = test_data_1_sub.apply(np.mean, axis=1)

test_data_1.sort_values(by=['average_map'], ascending= False,inplace=True)

	mean	std
mAP_test_abs_values_4	94.296667	1.122720
mAP_test_abs_values_17	94.182083	1.310986
mAP_test_abs_values_14	94.123958	0.930954
mAP_test_abs_values_5	93.846458	1.119767
mAP_test_abs_values_7	93.823750	1.397590
mAP_test_abs_values_13	93.802708	1.143032
mAP_test_abs_values_9	93.777917	1.039709
mAP_test_abs_values_16	93.701458	2.056142
mAP_test_abs_values_8	93.668542	1.776147
mAP_test_abs_values_18	93.499167	1.297836
mAP_test_abs_values	93.317083	1.754463
mAP_test_abs_values_15	93.155833	1.374008
mAP_test_abs_values_10	93.058542	6.741124
average_map	92.970192	0.896944
mAP_test_abs_values_23	92.909375	0.799450
mAP_test_abs_values_2	92.876250	6.748060
mAP_test_abs_values_21	92.700625	1.225030
mAP_test_abs_values_3	92.630208	6.714049
mAP_test_abs_values_12	92.420208	1.383664
mAP_test_abs_values_11	92.382292	1.258762
mAP_test_abs_values_26	92.251667	1.211190
mAP_test_abs_values_22	91.933333	6.539201
mAP_test_abs_values_6	91.848750	1.680716
mAP_test_abs_values_19	91.531042	6.493397
mAP_test_abs_values_25	91.520417	6.435556
mAP_test_abs_values_20	90.996458	6.417313

Summary using radar plot

def extract_number(text):
    if isinstance(text, str):
        matches = re.findall(r'\d+', text)
        return int(matches[0]) if matches else 1
    return 1

res_valid1['id'] = res_valid1.index.to_series().apply(extract_number)
res_test1['id'] = res_test1.index.to_series().apply(extract_number)



res_comb = pd.concat([res_valid1,res_test1])
index_series = res_comb.index.to_series()

res_comb['type'] = np.where(
    index_series.str.contains('valid', case=False), 'valid',
    np.where(index_series.str.contains('test', case=False), 'test', 'unknown')
)
res_comb = res_comb.query("type !='unknown'")

res_comb =  res_comb.reset_index(drop=True)


res_test1 = res_test1.sort_values(by=['id']).reset_index().query("index !='average_map'")
data_range1 = np.array(list(res_test1['mean']) + list(res_valid1['mean']))
categories = [str(i) for i in range(1,26)]
fig = go.Figure()

# Valid
fig.add_trace(go.Scatterpolar(
       r=list(res_test1['mean']),
      theta=categories,
      #fill='toself',
      name='Test'
))

# Test
fig.add_trace(go.Scatterpolar(
      r=list(res_valid1['mean']),
      theta=categories,
      #fill='toself',
      name='Valid'
))

# Customization of chart
fig.update_layout(
  title={
      'text': "T-Test",
        'xanchor': 'center',
      'yanchor': 'top'
  },
  polar=dict(
    radialaxis=dict(
      visible=True,
      range=[round(data_range1.min()), round(data_range1.max()) + 1]
    )),
  showlegend=True
)

fig.show()

##############


res_valid2['id'] = res_valid2.index.to_series().apply(extract_number)
res_test2['id'] = res_test2.index.to_series().apply(extract_number)



res_comb = pd.concat([res_valid2,res_test2])
index_series = res_comb.index.to_series()

res_comb['type'] = np.where(
    index_series.str.contains('valid', case=False), 'valid',
    np.where(index_series.str.contains('test', case=False), 'test', 'unknown')
)
res_comb = res_comb.query("type !='unknown'")

res_comb =  res_comb.reset_index(drop=True)


res_test2 = res_test2.sort_values(by=['id']).reset_index().query("index !='average_map'")

data_range2 = np.array(list(res_test2['mean']) + list(res_valid2['mean']))

categories = [str(i) for i in range(1,26)]
fig = go.Figure()

# Valid
fig.add_trace(go.Scatterpolar(
       r=list(res_test2['mean']),
      theta=categories,
      #fill='toself',
      name='Test'
))

# Test
fig.add_trace(go.Scatterpolar(
      r=list(res_valid2['mean']),
      theta=categories,
      #fill='toself',
      name='Valid'
))

# Customization of chart
fig.update_layout(
  title={
      'text': "MU-Test",
         'xanchor': 'center',
      'yanchor': 'top'
  },
  polar=dict(
    radialaxis=dict(
      visible=True,
      range=[round(data_range2.min()), round(data_range2.max()) + 1]
    )),
  showlegend=True
)

fig.show()

Experiment 2 Results

• In these experimental results, the thresholding is based on a fixed value of 0. This decision is informed by the fact that the binary-like hash values are symmetrically distributed between -1 and 1.

Validation Data mAP Results Preview:

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_zero	mAP_valid_zero_2	mAP_valid_zero_3	mAP_valid_zero_4	mAP_valid_zero_5	mAP_valid_zero_6	mAP_valid_zero_7	mAP_valid_zero_8	mAP_valid_zero_9	mAP_valid_zero_10	mAP_valid_zero_11	mAP_valid_zero_12	mAP_valid_zero_13	mAP_valid_zero_14	mAP_valid_zero_15	mAP_valid_zero_16	mAP_valid_zero_17	mAP_valid_zero_18	mAP_valid_zero_19	mAP_valid_zero_20	mAP_valid_zero_21	mAP_valid_zero_22	mAP_valid_zero_23	mAP_valid_zero_25	mAP_valid_zero_26
48	372	48	0.1	16	0.00100	24	0.000000e+00	0.000000e+00	84.33	84.25	86.57	85.57	85.37	86.02	84.52	86.70	83.63	86.06	85.84	87.70	86.80	84.25	84.93	85.19	82.44	85.28	88.69	90.66	88.10	85.45	88.59	86.61	87.02
51	372	48	0.1	16	0.00100	24	0.000000e+00	1.000000e-08	84.12	87.41	84.97	88.79	85.10	86.92	91.25	84.56	87.91	85.90	87.77	88.38	88.15	83.26	83.69	85.42	86.22	85.30	87.66	88.78	89.96	90.19	88.86	88.17	86.85
60	372	48	0.1	16	0.00100	24	1.000000e-08	0.000000e+00	84.86	86.14	85.22	85.95	82.35	43.72	83.56	89.27	84.94	87.89	88.35	86.37	43.72	43.72	86.52	86.69	86.03	43.72	88.45	87.46	88.17	89.31	86.90	88.41	89.28
63	372	48	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	84.33	84.87	84.94	85.44	85.05	87.47	85.72	88.76	85.19	84.17	87.83	87.04	86.83	87.52	85.46	88.03	84.29	86.88	86.75	89.62	87.93	86.21	88.39	87.51	87.70
112	372	48	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	85.46	84.82	85.39	84.50	87.67	87.58	82.54	87.31	86.38	84.12	87.60	87.59	87.09	86.31	87.60	88.18	85.11	87.47	88.37	86.89	89.43	88.57	88.19	87.78	88.73
115	372	48	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	84.11	85.80	86.76	84.18	85.98	86.80	85.68	87.43	85.80	85.78	86.77	88.39	87.55	85.61	85.55	85.68	86.56	85.58	87.26	90.42	89.22	87.51	89.67	88.66	88.75
124	372	48	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	86.26	87.10	86.70	85.74	85.73	87.02	85.71	86.90	87.49	85.10	87.92	87.98	88.11	88.09	87.95	87.56	86.75	87.22	88.51	88.11	89.44	88.82	89.68	89.00	88.74
127	372	48	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	84.52	84.96	86.31	86.87	87.52	87.96	85.68	85.63	88.25	87.08	87.59	89.82	88.50	87.64	88.62	87.52	86.43	86.94	88.51	88.28	87.82	88.94	87.86	88.06	88.85
176	372	48	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	87.68	86.69	43.72	84.53	83.73	86.66	85.94	87.61	86.84	87.96	87.53	88.18	86.48	85.56	87.26	86.63	86.36	87.18	87.39	86.94	89.36	89.13	84.66	88.02	87.17
179	372	48	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	84.84	87.46	84.84	83.88	85.38	88.80	82.85	85.42	86.83	85.91	86.37	88.40	87.77	87.30	88.11	86.79	86.45	83.87	89.05	90.28	88.32	87.25	90.12	88.52	89.05
188	372	48	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	84.51	88.84	86.31	85.26	86.95	87.67	86.64	88.40	87.31	88.42	88.58	87.57	87.37	83.06	84.12	89.56	85.67	87.02	88.59	90.20	89.16	88.69	90.84	88.18	88.10
191	372	48	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	86.23	87.70	85.03	84.92	87.99	87.59	85.17	84.59	87.20	84.65	87.54	87.08	87.47	87.10	85.65	87.35	84.85	86.86	87.24	90.19	86.33	88.70	88.37	88.66	90.41
240	372	48	0.1	16	0.00100	36	0.000000e+00	0.000000e+00	84.38	43.72	83.18	88.42	85.42	86.63	83.58	84.89	84.16	83.87	87.87	87.96	86.10	85.78	85.28	87.55	86.88	87.38	84.85	88.40	89.06	85.91	88.04	89.06	87.46
243	372	48	0.1	16	0.00100	36	0.000000e+00	1.000000e-08	82.91	86.34	87.16	87.24	84.69	86.55	85.07	86.38	85.90	84.43	85.54	85.59	87.86	84.64	86.39	85.43	86.01	85.92	87.86	91.04	87.26	88.32	87.93	86.59	87.95
252	372	48	0.1	16	0.00100	36	1.000000e-08	0.000000e+00	84.13	85.41	84.58	85.18	88.28	86.95	86.25	86.44	84.00	84.65	86.57	88.48	87.34	86.98	87.90	85.69	86.44	87.29	88.15	87.90	88.42	89.09	88.49	86.94	88.64
255	372	48	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	85.77	84.73	88.24	86.98	85.11	87.41	84.88	85.44	86.81	85.22	86.43	89.79	86.02	84.34	86.29	85.53	86.74	86.75	87.79	88.13	88.92	89.25	88.72	87.18	88.74
304	372	48	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	85.12	85.61	85.15	87.09	86.42	87.76	86.19	83.60	84.73	86.51	89.27	88.66	88.41	88.23	85.02	86.95	86.13	86.33	89.74	87.66	88.73	88.34	88.32	88.34	88.10
307	372	48	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	86.73	87.99	86.54	86.40	86.67	89.27	87.74	86.81	89.71	86.17	87.91	88.32	87.14	87.46	86.59	88.63	86.55	88.45	88.79	89.51	89.08	87.59	88.33	89.97	85.85
316	372	48	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	87.97	87.19	86.32	86.34	87.51	86.80	85.44	85.73	87.22	87.14	87.76	87.18	88.11	86.65	87.70	86.70	86.74	87.49	91.85	89.82	89.47	88.73	89.06	91.07	91.68
319	372	48	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	86.46	86.67	86.34	87.15	85.56	88.09	89.13	86.37	88.11	85.97	89.27	88.36	87.43	87.66	86.40	85.95	87.84	85.50	88.21	88.75	87.69	89.97	87.71	88.93	88.75
368	372	48	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	88.47	85.59	85.77	86.64	85.11	87.86	86.10	85.19	86.99	87.59	87.45	87.62	88.54	86.92	86.93	87.53	85.33	88.12	89.23	88.02	89.33	89.49	87.32	88.71	87.48
371	372	48	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	85.96	85.25	83.72	86.77	85.57	85.51	85.91	86.86	85.74	85.65	87.97	87.64	86.37	84.60	86.29	86.28	87.26	85.38	88.09	86.80	87.98	90.18	88.21	88.98	88.56
380	372	48	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	85.43	86.26	86.61	84.98	90.32	87.91	84.87	85.45	87.97	88.34	86.89	87.58	84.76	84.23	86.25	86.64	89.95	87.60	88.34	88.50	89.70	89.48	86.89	88.65	88.79
383	372	48	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	85.16	86.54	86.22	86.29	88.07	87.11	87.15	86.02	85.81	86.96	88.27	87.31	85.68	87.05	84.26	86.84	87.19	87.64	90.57	86.81	88.46	85.84	89.26	89.17	86.14
432	372	48	0.1	16	0.00100	48	0.000000e+00	0.000000e+00	84.27	86.36	88.12	84.72	83.41	84.43	84.53	84.93	84.69	85.00	87.54	86.68	86.27	85.35	88.36	85.61	85.31	85.74	87.54	87.64	88.80	87.43	88.45	87.77	87.52
435	372	48	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	85.30	87.35	86.43	85.20	88.81	86.82	87.23	85.86	88.58	88.08	88.12	88.15	87.68	85.70	86.47	88.28	85.28	88.04	90.15	87.30	90.03	89.23	88.94	87.35	88.70
444	372	48	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	84.73	87.20	86.51	87.78	86.15	86.81	84.49	85.33	86.52	85.39	87.51	86.81	88.53	86.67	86.50	86.03	86.90	85.02	43.72	87.27	88.31	90.62	88.99	43.72	88.11
447	372	48	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	85.28	85.62	85.13	85.87	85.94	85.06	82.56	86.13	86.26	43.72	86.65	88.17	84.64	86.72	86.51	87.00	87.07	88.50	87.44	43.72	86.65	89.89	88.46	90.89	87.82
496	372	48	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	85.03	85.29	84.55	84.75	86.94	88.15	85.02	86.32	87.68	85.94	87.82	88.84	88.83	87.16	83.06	87.61	87.52	86.46	88.24	89.97	89.95	88.67	89.61	90.14	88.58
499	372	48	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	86.99	85.53	85.61	83.12	85.06	87.86	86.91	86.57	86.33	87.58	88.76	89.36	86.64	84.92	87.18	87.61	85.17	86.70	89.10	88.90	88.97	89.36	89.57	89.54	86.34
508	372	48	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	88.05	86.54	87.80	86.96	85.97	87.13	87.32	86.47	86.02	86.03	90.38	88.90	89.56	86.16	86.25	87.55	85.76	86.38	88.53	88.91	88.51	86.70	87.90	88.25	90.45
511	372	48	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	85.91	86.23	86.04	89.41	85.57	87.50	86.67	85.45	86.99	84.16	86.21	89.38	86.20	86.04	88.09	86.47	86.76	86.10	89.25	88.42	87.14	89.30	86.49	87.55	85.61
560	372	48	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	85.94	87.79	85.65	81.72	83.75	88.05	86.36	87.41	87.48	84.82	86.99	88.01	85.81	84.73	89.45	90.01	88.42	85.84	85.12	86.36	89.02	89.55	87.54	88.88	87.30
563	372	48	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	84.76	84.80	84.89	87.77	86.08	87.23	83.26	88.28	85.67	87.60	89.48	87.90	88.12	90.53	87.26	86.75	87.23	85.58	87.11	85.54	89.79	89.23	87.03	89.24	87.70
572	372	48	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	87.56	83.76	85.15	86.51	86.79	88.01	85.14	87.21	86.49	85.88	86.14	88.37	87.21	89.92	88.13	84.97	84.55	85.69	90.87	87.96	90.19	89.28	88.20	87.85	88.95
575	372	48	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	84.76	84.87	86.55	87.13	84.09	84.93	87.10	85.24	86.73	84.21	87.29	85.09	86.63	89.51	89.72	86.59	88.82	87.59	87.12	89.29	88.23	87.26	86.52	89.74	88.94
624	372	48	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	82.70	84.27	83.80	88.04	84.41	86.85	83.31	87.25	84.46	84.71	91.23	86.76	85.16	86.21	86.66	83.93	85.30	87.14	88.51	88.10	88.60	87.08	88.70	88.48	86.64
627	372	48	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	83.61	85.30	85.72	86.94	86.73	86.10	86.02	87.60	86.85	85.28	86.92	86.30	87.94	85.53	87.05	87.10	85.95	86.65	87.82	86.64	89.37	88.48	88.50	88.61	87.50
636	372	48	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	85.96	85.28	85.02	84.01	84.06	85.61	84.53	86.26	85.32	83.96	89.52	87.64	86.39	87.52	85.96	86.74	85.52	86.74	88.85	88.02	89.87	89.04	90.41	85.79	89.04
639	372	48	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	85.07	85.76	85.19	85.67	84.40	85.36	82.07	85.73	84.11	84.46	85.15	86.64	85.48	87.56	87.72	84.02	87.72	85.97	87.23	86.77	88.29	87.48	88.23	88.11	87.16
688	372	48	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	85.06	87.77	86.19	85.57	82.79	86.48	86.11	85.73	89.92	89.08	87.51	87.59	86.91	86.70	88.27	88.63	83.80	87.46	88.80	87.71	88.43	88.27	87.50	88.82	86.32
691	372	48	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	87.36	86.62	87.05	86.32	88.37	89.31	86.40	89.78	87.17	85.77	89.62	88.19	87.03	88.06	87.65	87.47	86.34	86.24	90.23	89.33	87.93	89.43	89.24	90.10	89.04
700	372	48	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	86.71	86.71	86.92	85.97	85.74	86.88	85.93	86.52	87.18	87.88	90.11	86.12	86.22	88.31	86.42	87.52	85.41	87.19	89.37	87.38	87.49	88.63	89.05	89.67	88.62
703	372	48	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	87.11	87.52	87.10	87.61	86.63	85.83	90.15	85.42	88.01	87.42	88.43	87.34	85.89	85.23	85.97	86.84	85.83	84.70	89.43	88.96	88.53	88.60	88.02	87.92	89.43
752	372	48	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	85.56	84.73	84.70	82.51	82.57	86.40	87.50	85.86	85.17	85.08	86.69	87.51	88.32	87.28	86.23	85.13	85.09	87.81	86.45	89.74	89.05	88.82	86.89	85.27	87.09
755	372	48	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	87.14	89.30	85.99	86.06	87.95	87.45	87.32	89.24	87.85	85.83	89.17	89.72	88.81	87.38	85.47	85.87	86.34	87.83	89.34	89.41	90.70	89.07	87.53	89.91	89.64
764	372	48	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	85.85	85.18	86.20	84.52	88.04	87.21	87.27	87.62	85.96	86.23	87.90	86.87	86.26	86.08	88.10	88.56	89.16	87.04	89.06	89.77	90.80	88.02	89.34	89.59	87.75
767	372	48	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	85.73	85.20	86.07	85.96	83.75	87.67	84.45	86.95	87.15	87.44	87.36	86.44	86.03	87.90	85.84	86.12	85.67	85.38	87.18	87.80	88.46	88.38	89.50	89.44	87.65
816	372	48	0.1	16	0.00010	36	0.000000e+00	0.000000e+00	84.32	83.61	83.80	87.43	85.15	84.83	86.34	85.39	86.40	85.83	85.99	86.01	85.87	87.06	87.33	85.75	86.44	86.88	89.90	87.72	88.54	87.23	88.64	87.91	87.35
819	372	48	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	84.13	85.97	84.67	85.26	84.67	89.50	84.51	84.86	84.89	85.61	86.03	86.91	85.69	86.16	87.71	87.19	85.54	86.23	89.31	87.08	87.67	88.50	87.67	88.03	87.86
828	372	48	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	84.65	85.29	86.67	85.14	84.31	85.94	83.14	88.15	85.95	85.30	89.17	88.61	87.09	87.89	86.47	87.29	86.58	85.49	90.36	90.59	88.97	89.85	88.61	88.12	85.78
831	372	48	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	84.61	85.53	84.05	87.34	86.36	87.33	85.49	87.80	85.35	85.72	88.35	85.94	87.97	86.71	89.26	86.50	86.23	84.62	89.72	87.65	87.77	88.58	88.69	88.23	88.32
880	372	48	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	86.79	85.23	85.11	84.48	86.70	87.89	82.52	85.87	89.26	85.38	88.76	86.00	84.18	84.84	87.60	89.08	86.91	87.55	88.30	87.48	88.20	86.79	87.37	89.00	89.54
883	372	48	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	88.05	43.72	88.02	87.19	86.27	87.97	84.72	86.70	87.01	87.58	86.03	89.18	85.49	84.13	85.60	87.93	86.96	86.73	87.70	87.58	86.80	43.72	89.23	87.85	88.42
892	372	48	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	86.75	87.43	87.24	85.47	86.19	87.64	86.70	88.84	87.88	87.36	88.78	86.86	88.07	86.63	86.45	87.89	86.79	85.95	88.77	89.68	88.42	88.18	89.49	88.68	89.25
895	372	48	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	85.45	88.30	85.38	87.46	87.46	86.33	84.59	86.11	86.63	87.51	88.73	87.45	89.01	87.14	87.18	88.87	86.46	87.18	87.81	89.28	89.36	89.51	89.02	90.31	88.78
944	372	48	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	84.95	85.13	89.58	85.79	86.51	87.29	85.46	88.35	86.87	87.17	87.23	86.33	88.43	86.39	86.41	87.30	87.19	85.54	87.89	87.70	89.80	88.93	88.68	89.82	88.10
947	372	48	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	86.70	86.14	86.80	87.92	85.78	86.78	87.64	87.65	85.68	86.71	87.50	87.14	85.66	84.69	88.11	84.86	85.86	85.62	87.36	89.50	88.36	88.51	88.35	86.80	88.86
956	372	48	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	87.01	86.13	85.96	85.02	85.81	86.60	88.60	88.60	87.16	89.29	88.90	89.14	85.89	87.47	85.91	87.36	85.82	86.88	88.67	87.47	86.83	88.92	87.65	87.57	86.79
959	372	48	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	85.16	84.62	85.88	86.08	86.27	88.06	86.05	86.27	85.79	86.90	86.54	89.15	86.79	85.70	90.50	87.77	83.68	86.61	88.92	88.56	90.29	89.12	88.87	86.43	88.12
1008	372	48	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	85.01	86.32	85.19	86.40	83.92	87.04	86.35	85.92	87.63	85.40	87.62	91.17	87.71	85.61	88.14	87.68	86.91	87.58	89.84	88.17	89.09	88.26	87.00	89.31	85.99
1011	372	48	0.1	16	0.00010	48	0.000000e+00	1.000000e-08	85.40	87.35	86.54	84.63	84.49	88.20	85.85	86.37	86.43	85.35	86.02	84.46	86.55	83.91	85.17	86.49	86.18	86.27	87.53	87.55	88.38	89.56	88.57	89.85	87.88
1020	372	48	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	83.73	86.37	86.35	89.26	85.23	87.49	85.22	87.35	86.61	89.02	88.36	87.91	86.51	85.40	86.66	86.47	86.87	87.63	88.56	89.30	89.08	89.62	88.82	89.46	88.28
1023	372	48	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	84.31	86.21	86.73	87.96	85.71	90.44	87.60	86.32	87.00	88.56	86.66	89.55	87.54	86.04	86.97	88.76	86.68	86.46	88.02	87.06	88.84	87.87	89.03	87.84	87.02
1072	372	48	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	83.42	84.47	87.05	86.85	88.44	87.66	84.77	87.19	86.75	87.69	87.66	87.61	87.77	86.70	85.08	85.53	85.26	86.47	89.05	90.21	89.88	86.18	87.95	89.14	88.75
1075	372	48	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	85.87	85.27	84.84	85.99	88.01	87.73	87.68	86.52	88.28	86.04	87.69	88.04	86.14	86.80	85.35	88.91	85.01	86.67	89.85	88.34	89.38	87.77	87.84	87.45	89.35
1084	372	48	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	88.63	88.33	87.01	86.57	89.05	88.56	87.57	87.38	89.00	86.30	87.32	89.44	87.40	88.07	86.77	89.25	85.38	87.85	87.80	89.60	89.31	88.06	87.94	89.04	90.95
1087	372	48	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	86.25	85.16	85.74	85.14	85.78	85.25	85.66	86.45	87.11	85.69	88.98	86.78	87.89	86.46	86.52	88.04	86.46	86.59	88.05	89.49	89.42	88.45	87.67	87.28	88.47
1136	372	48	0.1	64	0.00010	48	0.000000e+00	0.000000e+00	83.09	86.29	85.57	86.00	85.64	87.40	85.05	87.94	84.60	86.62	88.61	84.68	86.77	88.28	85.14	88.57	87.85	86.96	87.69	86.76	79.64	89.24	88.30	89.15	89.68
1139	372	48	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	87.46	84.09	86.73	84.85	86.78	88.28	83.86	87.93	87.85	85.94	85.68	85.76	84.95	87.79	86.44	89.09	86.41	85.77	86.80	87.83	85.83	88.20	88.37	88.36	87.26
1148	372	48	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	86.32	86.40	86.04	84.35	82.01	84.48	85.70	84.94	87.77	86.59	89.17	89.03	84.57	84.08	84.50	86.89	86.30	85.23	86.97	87.65	87.50	90.65	87.76	90.48	87.71
1151	372	48	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	86.06	86.47	86.56	88.02	84.68	86.86	87.41	89.53	86.65	87.54	87.58	88.94	88.64	86.36	86.17	87.12	85.26	86.82	87.66	87.84	89.05	89.52	89.82	89.19	88.31
1200	372	48	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	83.45	85.10	84.45	89.39	86.73	86.24	84.27	86.20	85.47	84.07	87.57	87.94	85.04	85.05	86.03	85.60	86.91	85.39	87.72	87.08	88.91	87.81	87.56	87.89	87.30
1203	372	48	0.1	16	0.00001	24	0.000000e+00	1.000000e-08	85.27	86.14	85.89	88.15	86.25	86.84	83.05	85.98	87.08	84.70	87.54	88.21	86.14	85.71	88.03	86.93	87.48	86.49	87.00	89.14	88.86	88.32	88.22	87.63	87.64
1212	372	48	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	85.33	87.10	85.61	85.53	85.11	86.70	85.83	87.46	88.08	85.44	89.38	87.39	86.52	86.25	86.46	86.00	86.18	85.19	87.98	89.57	86.26	87.26	88.45	88.36	88.29
1215	372	48	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	84.31	84.89	84.41	85.16	84.01	85.15	82.38	88.33	81.81	84.81	87.18	88.43	89.24	86.17	85.48	85.08	85.59	86.48	86.67	86.45	89.29	85.87	86.52	88.50	88.80
1264	372	48	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	85.42	86.55	84.12	84.32	83.09	87.55	83.44	88.90	87.16	87.19	87.49	87.67	86.31	85.86	85.88	88.16	86.14	86.15	88.10	90.32	90.09	87.09	86.69	88.57	88.91
1267	372	48	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	85.03	86.22	86.06	86.73	86.93	86.84	86.89	85.37	86.60	87.03	88.65	87.78	86.82	88.91	87.20	85.12	88.18	86.72	88.10	88.61	89.38	90.10	89.42	88.81	87.92
1276	372	48	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	85.24	85.31	86.48	87.06	87.37	86.50	87.60	86.88	86.03	85.51	88.64	87.50	86.32	86.46	86.86	86.19	88.12	88.78	88.59	87.53	89.51	88.74	85.83	89.44	89.70
1279	372	48	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	86.12	87.66	85.82	84.33	87.44	88.13	86.64	85.14	87.75	86.36	87.75	88.15	86.77	84.66	87.00	87.18	87.30	85.05	88.79	88.20	88.87	89.29	89.03	88.65	89.16
1328	372	48	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	85.15	88.47	85.95	62.08	85.23	86.69	86.90	84.74	87.74	85.66	86.82	89.57	88.13	87.45	87.23	86.96	87.28	84.39	87.78	88.95	87.93	88.00	89.16	85.87	83.09
1331	372	48	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	86.29	84.50	86.38	85.46	86.26	86.51	85.90	85.16	91.28	86.67	87.26	87.69	87.80	86.82	87.08	87.68	86.94	86.89	88.53	88.61	88.70	86.83	88.26	89.02	88.51
1340	372	48	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	85.83	85.42	85.28	86.84	85.37	87.83	85.21	89.57	85.70	86.23	86.49	87.10	86.56	88.86	87.27	88.28	86.03	86.35	88.17	87.33	89.33	87.90	85.35	89.23	87.44
1343	372	48	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	84.12	86.72	86.05	87.04	86.00	88.05	86.78	87.75	85.80	86.81	90.25	88.48	89.36	87.58	84.67	87.21	84.59	86.55	87.36	88.03	89.23	89.10	87.74	88.93	89.25
1392	372	48	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	83.67	87.20	86.32	85.24	86.04	86.72	86.72	89.13	89.22	85.78	86.08	88.47	87.48	86.01	88.10	87.25	86.70	86.39	87.37	87.09	90.09	89.14	89.56	89.38	90.02
1395	372	48	0.1	16	0.00001	36	0.000000e+00	1.000000e-08	83.68	87.11	85.82	88.38	84.69	86.67	85.28	85.18	84.62	85.92	88.44	88.78	86.52	82.78	86.43	85.98	85.04	86.92	90.42	88.11	88.27	89.58	88.21	89.10	88.39
1404	372	48	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	85.73	84.30	85.69	89.06	84.72	87.60	85.70	85.67	85.88	86.20	87.57	87.22	86.34	86.18	86.01	87.29	84.38	89.27	87.06	88.10	87.90	86.45	89.00	87.99	86.21
1407	372	48	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	84.51	86.00	86.58	84.79	84.54	86.20	86.28	90.42	85.11	85.21	88.05	87.61	87.08	87.68	88.08	87.36	85.01	85.72	87.25	87.06	89.76	88.99	89.82	87.78	90.18
1456	372	48	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	87.95	88.06	85.77	86.94	84.89	84.84	85.72	90.33	84.09	85.96	87.61	89.63	86.48	86.65	85.43	86.41	85.54	86.09	88.69	89.12	87.57	89.69	88.75	90.49	86.91
1459	372	48	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	86.24	86.99	87.10	85.66	86.23	86.45	85.97	86.53	86.10	87.29	89.29	86.05	86.17	87.64	84.36	86.73	85.87	87.37	87.73	87.66	89.37	89.90	90.76	88.51	89.61
1468	372	48	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	87.24	86.62	86.06	85.13	87.90	88.96	86.40	84.54	88.66	86.36	88.23	87.04	87.20	87.12	87.26	88.77	88.15	86.60	91.25	87.59	89.03	88.63	89.19	88.59	89.80
1471	372	48	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	86.10	86.75	86.95	90.66	84.58	86.47	87.87	86.76	84.15	86.43	88.70	89.32	89.73	83.03	86.03	87.96	85.29	88.96	88.65	91.07	88.91	89.02	90.18	88.62	87.05
1520	372	48	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	84.70	84.38	85.96	87.38	86.13	88.15	86.74	87.18	86.23	87.70	89.02	86.91	87.33	84.89	86.05	88.13	88.09	87.59	87.23	88.21	88.98	87.57	88.23	89.92	89.03
1523	372	48	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	86.87	85.51	85.05	87.08	89.28	85.70	86.90	86.01	85.99	86.86	86.40	91.48	87.24	87.75	86.45	86.02	84.76	85.59	91.21	88.24	86.02	88.99	89.52	89.48	87.13
1532	372	48	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	85.06	86.28	85.21	85.45	86.48	87.36	84.60	88.29	85.36	86.95	87.13	87.69	85.40	88.22	84.28	89.38	85.85	86.56	87.51	88.81	88.82	86.84	89.26	87.30	88.22
1535	372	48	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	85.42	87.50	88.68	86.42	86.05	86.21	86.95	85.50	86.01	85.47	87.91	88.15	87.77	86.70	86.32	87.15	89.90	86.77	88.74	92.23	88.88	88.54	88.37	88.51	88.18

Test Data mAP Results Preview:

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_zero	mAP_test_zero_2	mAP_test_zero_3	mAP_test_zero_4	mAP_test_zero_5	mAP_test_zero_6	mAP_test_zero_7	mAP_test_zero_8	mAP_test_zero_9	mAP_test_zero_10	mAP_test_zero_11	mAP_test_zero_12	mAP_test_zero_13	mAP_test_zero_14	mAP_test_zero_15	mAP_test_zero_16	mAP_test_zero_17	mAP_test_zero_18	mAP_test_zero_19	mAP_test_zero_20	mAP_test_zero_21	mAP_test_zero_22	mAP_test_zero_23	mAP_test_zero_25	mAP_test_zero_26
48	372	48	0.1	16	0.00100	24	0.000000e+00	0.000000e+00	91.62	93.12	92.33	91.84	91.78	91.42	89.05	90.49	93.82	91.27	91.53	90.07	92.36	93.47	90.92	92.12	92.59	92.38	89.73	89.63	92.17	90.26	90.67	92.51	89.58
51	372	48	0.1	16	0.00100	24	0.000000e+00	1.000000e-08	93.79	92.93	96.39	91.31	94.47	92.76	91.84	94.65	94.78	93.42	90.98	92.95	93.33	94.55	93.10	92.96	94.33	94.85	94.13	90.11	93.62	92.76	92.07	92.27	92.44
60	372	48	0.1	16	0.00100	24	1.000000e-08	0.000000e+00	92.04	93.08	93.96	93.10	94.05	48.12	93.61	93.61	93.12	92.61	93.64	88.43	48.12	48.12	92.96	93.17	92.05	48.12	94.37	91.22	93.02	92.83	90.52	91.08	93.16
63	372	48	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	91.37	92.68	95.02	93.76	91.19	93.83	92.94	92.98	91.49	92.73	91.69	91.54	93.40	91.72	93.15	93.40	93.44	94.95	94.35	92.57	93.60	93.22	93.61	92.06	92.10
112	372	48	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	92.45	94.40	95.57	90.83	94.10	92.80	94.79	94.14	94.57	94.69	93.86	91.50	94.00	93.79	94.38	93.29	94.42	93.27	92.96	93.74	93.24	92.23	91.82	92.55	92.88
115	372	48	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	92.25	92.48	93.88	93.85	93.59	92.70	92.25	91.89	93.81	92.53	92.25	93.75	92.15	92.60	92.79	93.54	93.98	94.14	93.50	91.10	92.20	92.19	92.89	93.71	91.58
124	372	48	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	90.62	94.57	91.43	93.95	94.12	91.66	93.74	94.79	92.49	93.74	89.82	91.57	94.92	93.28	92.72	92.39	93.83	94.24	93.62	91.31	91.83	91.93	92.06	93.30	93.10
127	372	48	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	92.52	92.13	94.57	93.75	95.07	92.36	93.22	92.93	95.74	93.27	94.20	90.88	94.36	96.10	89.80	93.70	94.98	93.53	94.42	92.53	92.41	92.38	90.09	91.67	89.05
176	372	48	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	93.70	90.77	48.12	92.68	93.50	90.95	93.76	95.68	93.71	95.41	94.35	92.85	93.28	95.18	93.97	93.11	91.94	93.59	92.56	92.82	89.16	93.12	91.32	92.50	91.88
179	372	48	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	93.83	92.33	94.42	94.71	93.04	93.44	92.79	93.65	93.51	94.87	92.83	94.43	95.88	95.05	93.43	94.67	94.90	95.83	91.99	91.40	92.58	93.77	94.42	91.17	93.80
188	372	48	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	94.18	92.37	92.74	91.60	93.47	92.76	95.05	94.40	93.53	94.32	91.30	89.92	94.38	90.84	94.74	93.32	95.51	94.95	93.46	89.46	93.61	91.50	94.72	92.34	88.71
191	372	48	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	95.45	95.32	93.71	94.66	94.81	93.26	94.94	92.30	94.45	92.96	94.53	92.14	93.05	94.79	95.92	91.75	95.29	92.56	91.74	90.64	93.44	92.09	93.72	92.41	92.95
240	372	48	0.1	16	0.00100	36	0.000000e+00	0.000000e+00	90.85	48.12	93.81	94.55	91.63	90.96	91.56	91.09	91.87	91.28	91.84	91.55	92.89	93.80	92.79	92.75	90.85	93.92	89.83	92.52	91.03	90.67	93.69	90.17	90.28
243	372	48	0.1	16	0.00100	36	0.000000e+00	1.000000e-08	93.64	91.79	92.43	94.00	93.20	93.30	94.16	93.83	93.64	92.82	93.71	91.60	94.06	92.38	95.69	94.35	93.49	94.93	91.83	90.75	91.24	93.64	93.81	95.58	92.66
252	372	48	0.1	16	0.00100	36	1.000000e-08	0.000000e+00	92.25	95.66	94.35	95.08	96.12	92.22	92.57	96.26	96.11	93.91	96.08	91.79	94.14	94.25	93.98	94.67	94.46	94.75	92.37	91.79	94.47	92.98	93.86	94.08	93.68
255	372	48	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	93.31	91.41	93.40	94.39	95.17	93.78	91.33	94.81	94.04	93.72	93.97	93.50	94.18	94.37	94.96	94.56	95.27	95.12	93.57	91.46	92.47	91.53	93.58	94.55	90.26
304	372	48	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	94.39	92.08	95.38	94.40	92.78	94.01	94.48	95.20	94.47	95.03	92.89	90.93	94.75	93.59	91.18	93.83	92.26	94.94	90.78	92.14	92.57	94.28	92.90	92.92	90.11
307	372	48	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	93.38	94.54	91.04	95.16	92.55	89.78	95.01	94.57	94.24	93.75	93.06	91.70	95.36	93.01	92.31	92.70	95.06	94.65	91.22	92.35	93.35	91.46	94.07	91.11	90.75
316	372	48	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	93.60	95.97	93.82	95.03	94.34	91.42	96.37	93.64	94.94	93.94	92.39	94.10	92.99	94.44	92.59	93.36	94.86	91.57	92.36	91.99	93.16	93.25	92.37	93.11	92.66
319	372	48	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	94.23	91.41	91.90	95.00	94.48	91.07	94.07	94.24	93.01	93.05	92.59	94.00	94.57	94.54	94.39	95.55	91.52	92.03	92.53	93.29	93.69	93.70	91.93	92.25	91.78
368	372	48	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	92.53	93.44	94.70	92.93	95.77	92.26	94.44	95.39	94.72	94.23	94.71	93.61	94.64	93.80	94.13	93.21	94.80	94.27	93.03	93.46	91.82	92.08	92.65	93.93	91.98
371	372	48	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	93.98	93.96	94.46	95.37	95.78	91.97	93.43	95.06	95.20	94.29	94.14	93.52	94.16	93.44	94.82	95.61	94.97	94.05	93.41	92.43	94.81	92.45	94.60	95.18	91.01
380	372	48	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	94.19	93.06	93.25	93.59	92.99	92.59	94.38	94.41	93.87	93.41	91.64	92.16	97.07	97.09	92.35	93.87	93.79	95.72	92.72	92.77	92.72	90.75	93.41	92.16	91.17
383	372	48	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	94.90	94.87	92.87	92.39	95.89	92.02	95.08	93.63	93.88	95.40	92.59	92.94	92.17	95.53	94.81	93.43	94.01	92.38	90.91	91.57	94.63	93.45	92.11	93.74	93.61
432	372	48	0.1	16	0.00100	48	0.000000e+00	0.000000e+00	91.00	94.56	91.93	94.41	95.27	92.93	94.49	91.38	91.87	96.23	91.71	93.68	94.17	94.91	93.98	93.64	92.59	92.71	93.91	89.95	88.79	93.80	92.14	91.76	91.84
435	372	48	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	93.23	95.13	93.87	93.51	93.99	91.78	92.51	94.02	94.74	94.74	94.23	94.73	93.46	92.87	94.22	95.55	94.09	93.76	92.65	89.88	94.25	91.83	92.81	91.55	93.28
444	372	48	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	93.79	94.99	93.71	94.28	94.21	90.24	93.45	95.98	94.81	93.07	93.14	91.27	94.35	94.68	94.15	92.62	95.22	94.52	48.12	91.77	93.15	91.15	93.79	48.12	92.43
447	372	48	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	94.84	94.85	95.33	94.95	95.47	92.66	93.35	95.54	94.83	48.12	93.80	93.65	95.52	95.06	96.34	93.75	96.51	94.30	93.97	48.12	94.90	91.74	92.62	92.89	91.84
496	372	48	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	92.61	93.44	93.05	92.75	94.01	92.72	92.37	94.55	94.04	93.62	91.18	91.52	93.63	93.32	94.16	93.75	95.24	94.09	93.52	92.41	92.92	93.93	94.50	93.23	93.49
499	372	48	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	94.16	95.80	94.15	93.46	93.07	92.44	91.87	94.40	94.42	93.42	94.29	93.33	92.08	93.99	95.05	92.34	95.92	91.79	91.93	91.92	94.02	87.90	93.58	93.21	93.06
508	372	48	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	93.90	95.05	92.75	94.51	93.55	92.41	94.31	94.22	95.80	94.24	91.74	93.11	92.54	94.02	94.12	92.65	96.25	94.58	92.82	92.77	94.30	93.41	93.92	94.80	92.14
511	372	48	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	94.77	93.07	95.58	92.96	94.79	91.08	94.34	95.53	91.12	93.71	91.74	93.62	91.29	94.64	93.13	92.78	93.72	95.38	91.53	91.44	92.44	92.50	91.90	93.76	93.25
560	372	48	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	92.51	94.13	94.52	90.97	89.13	92.29	92.28	93.77	91.58	95.01	92.27	93.34	90.59	94.04	94.50	93.74	94.55	92.95	92.36	92.89	91.75	92.25	91.62	93.73	92.33
563	372	48	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	94.52	93.96	92.93	93.47	94.86	92.80	94.38	94.78	93.05	93.93	92.12	91.73	93.64	93.60	94.88	93.83	95.19	95.33	92.34	93.36	92.72	94.29	92.97	91.94	93.21
572	372	48	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	95.76	95.70	91.35	91.46	94.63	93.07	94.32	94.91	92.85	94.68	92.22	92.80	93.83	95.03	93.44	94.59	95.10	95.17	92.82	92.80	92.68	92.72	91.81	93.41	90.82
575	372	48	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	93.33	91.31	92.90	92.81	93.79	92.24	94.62	92.19	92.10	93.96	92.87	92.62	94.05	94.87	93.29	93.76	93.98	93.75	91.03	90.73	91.00	91.78	92.30	93.52	90.86
624	372	48	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	92.58	89.80	93.06	91.02	92.86	91.11	93.11	93.92	93.06	90.54	90.34	93.16	90.68	94.10	94.04	89.23	93.61	89.77	90.95	89.25	93.53	93.10	91.92	91.83	89.56
627	372	48	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	92.13	92.87	91.80	92.50	91.90	91.47	92.64	94.66	93.35	95.26	92.39	92.57	93.59	94.51	93.48	94.13	95.29	91.64	91.44	87.32	93.07	90.15	93.07	89.59	90.65
636	372	48	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	93.72	94.01	95.29	92.43	87.11	93.38	91.85	92.08	90.73	94.23	91.38	88.86	94.58	94.60	93.72	92.21	93.54	91.57	92.36	89.73	92.48	93.57	90.56	92.99	92.27
639	372	48	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	91.49	94.72	93.75	93.15	91.99	94.35	92.06	90.84	93.60	92.99	93.80	92.86	96.22	91.27	93.13	94.62	90.61	90.84	92.49	89.86	91.05	93.34	92.03	94.21	92.24
688	372	48	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	90.49	92.91	91.19	95.70	91.55	91.81	90.92	93.99	90.86	91.77	89.47	91.32	92.83	93.61	91.31	93.30	93.63	89.03	92.10	92.11	92.06	92.79	92.78	92.82	91.21
691	372	48	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	92.56	93.20	95.29	93.18	92.99	91.37	94.41	93.20	94.32	90.86	93.95	90.54	92.71	93.35	92.87	93.86	96.18	92.60	91.94	93.24	91.32	93.00	93.09	91.37	93.08
700	372	48	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	94.65	92.82	93.94	93.24	90.52	92.30	92.67	91.57	94.57	91.77	94.14	92.60	93.87	92.77	93.62	94.36	93.30	94.60	92.00	92.61	92.86	92.81	92.70	93.89	93.05
703	372	48	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	83.77	93.02	95.00	94.53	95.51	93.75	92.66	95.00	95.04	95.54	92.81	94.04	95.04	96.19	93.64	94.31	94.31	93.97	94.14	92.27	94.22	92.93	91.82	92.53	93.92
752	372	48	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	93.05	94.46	92.46	93.47	94.87	92.45	95.95	94.78	93.60	95.81	91.03	91.15	93.84	95.20	94.40	92.95	94.21	92.49	92.93	93.14	93.55	92.69	93.05	91.99	94.32
755	372	48	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	93.61	93.12	93.15	95.64	93.75	91.55	94.29	95.06	94.69	93.61	89.31	91.46	93.41	94.24	92.63	94.38	95.42	95.06	93.07	92.34	90.95	91.58	93.51	92.57	91.64
764	372	48	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	93.48	95.77	95.11	93.67	92.00	93.08	93.46	94.52	94.74	95.38	92.83	93.64	94.15	94.13	91.96	95.01	94.00	94.14	92.87	91.59	92.54	93.34	93.11	93.46	92.97
767	372	48	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	93.80	94.13	94.10	94.51	95.42	88.63	94.41	95.14	92.36	94.10	93.54	93.61	91.99	93.13	94.25	95.30	94.66	93.98	93.09	92.56	94.03	92.50	92.19	93.17	92.27
816	372	48	0.1	16	0.00010	36	0.000000e+00	0.000000e+00	93.69	92.36	94.49	91.29	94.69	90.96	92.91	94.32	93.03	92.47	91.07	91.60	91.42	93.33	92.41	93.62	94.52	90.13	91.86	88.54	94.09	91.86	93.19	91.94	91.57
819	372	48	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	91.29	91.99	91.43	94.57	95.01	90.15	91.79	94.91	92.43	93.78	90.83	95.36	92.11	94.51	92.19	94.37	93.87	95.35	89.37	92.00	93.23	92.39	93.42	92.68	88.95
828	372	48	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	94.15	94.89	94.60	93.15	94.63	92.69	93.90	95.30	95.13	95.79	92.11	91.28	93.11	93.72	93.83	94.94	93.50	93.38	92.98	91.56	93.08	92.70	93.38	92.98	92.77
831	372	48	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	94.24	95.28	95.42	95.97	95.39	92.12	92.32	94.50	94.90	93.29	90.99	93.20	94.22	94.32	92.77	94.49	94.90	94.86	92.92	92.36	93.65	95.77	94.34	93.01	93.88
880	372	48	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	92.37	92.88	92.78	93.26	93.87	91.27	92.76	93.14	91.91	94.97	91.88	92.55	94.69	93.24	92.87	92.71	92.56	92.92	94.31	90.42	92.69	90.94	92.52	91.51	91.83
883	372	48	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	95.11	48.12	92.83	92.39	94.44	92.91	93.37	92.19	92.35	94.78	92.83	93.52	93.87	95.26	94.74	95.20	94.83	95.32	91.41	89.77	93.12	48.12	93.12	91.84	92.18
892	372	48	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	93.34	91.70	92.76	95.52	94.60	93.56	92.59	89.66	92.92	94.74	90.15	90.52	91.21	94.28	94.80	93.85	94.34	92.68	92.15	92.25	90.14	92.90	92.70	92.15	93.40
895	372	48	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	93.70	94.89	92.13	92.22	94.31	92.87	92.55	93.17	94.58	92.86	90.26	91.63	91.28	93.82	92.72	92.85	94.41	93.53	91.30	93.39	90.67	93.59	92.90	91.86	91.81
944	372	48	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	94.74	94.33	93.38	95.63	95.22	94.09	95.05	95.23	95.06	94.59	92.54	93.23	94.30	95.02	91.99	94.46	94.85	92.67	91.98	90.35	93.54	95.18	94.39	92.19	93.28
947	372	48	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	93.86	93.94	95.74	93.23	93.09	93.37	92.18	92.94	92.05	93.20	91.61	93.58	93.94	94.23	95.14	93.54	95.58	93.42	92.59	92.70	92.79	91.39	92.96	93.00	93.73
956	372	48	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	93.45	93.32	92.03	94.89	93.06	91.31	93.31	91.71	93.79	92.18	91.19	92.41	93.41	95.07	93.59	94.27	96.05	92.30	94.01	92.87	91.43	95.60	92.89	93.16	93.05
959	372	48	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	92.78	95.67	95.50	95.80	94.38	93.63	96.40	94.32	94.58	95.30	93.42	92.88	95.90	95.60	92.79	95.37	95.69	92.68	94.14	93.38	93.41	93.55	92.48	93.43	92.79
1008	372	48	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	94.29	93.73	93.97	96.45	94.51	92.07	93.22	95.15	92.52	94.63	93.10	91.15	94.64	94.60	93.41	94.92	94.53	95.70	92.97	94.03	93.14	94.55	93.77	92.93	93.84
1011	372	48	0.1	16	0.00010	48	0.000000e+00	1.000000e-08	92.07	96.09	94.17	95.66	94.90	91.34	94.19	95.55	93.28	94.21	93.74	93.67	92.65	96.14	95.99	95.22	95.55	93.40	93.48	93.35	93.60	93.75	93.84	93.20	93.67
1020	372	48	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	93.96	95.67	93.92	94.53	93.34	92.91	95.05	92.34	92.96	95.16	92.88	91.60	92.47	93.61	93.45	93.16	93.49	92.94	93.48	92.31	90.62	92.68	93.93	92.42	92.49
1023	372	48	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	93.27	94.54	93.51	95.79	93.38	83.27	94.11	95.16	94.71	94.29	93.64	95.32	93.90	94.54	90.75	94.23	95.02	94.36	92.18	91.87	93.43	92.53	91.95	93.12	92.78
1072	372	48	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	92.47	94.86	94.88	95.79	94.55	91.69	91.07	91.07	93.07	92.99	92.80	93.93	94.78	94.47	93.28	94.48	92.70	94.20	92.43	92.72	93.26	94.93	92.54	92.19	93.18
1075	372	48	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	93.73	94.84	94.50	93.12	96.27	91.99	93.29	92.64	93.00	95.93	92.67	93.29	93.46	94.45	93.47	91.38	94.11	95.89	94.52	91.76	92.09	93.14	91.15	93.65	91.56
1084	372	48	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	93.87	92.20	93.08	95.04	94.50	92.63	91.91	92.63	94.57	95.36	91.69	91.67	94.30	94.98	92.32	95.25	96.03	92.80	92.56	91.48	92.74	93.31	91.53	92.22	93.46
1087	372	48	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	92.81	93.56	93.97	95.14	94.50	92.16	94.41	93.54	93.18	93.85	92.13	91.16	94.73	93.55	94.46	96.50	94.67	93.85	92.96	93.84	93.27	92.44	93.76	93.02	91.01
1136	372	48	0.1	64	0.00010	48	0.000000e+00	0.000000e+00	96.31	93.41	91.29	91.88	94.64	92.54	93.58	91.64	95.43	92.78	92.59	93.21	94.42	95.07	94.48	92.06	94.10	92.32	93.07	92.96	90.76	93.04	91.51	91.09	91.49
1139	372	48	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	94.38	94.97	94.38	94.00	94.40	93.33	93.33	94.56	95.31	94.41	93.51	92.93	95.01	93.02	95.86	94.28	94.55	94.15	91.36	93.13	93.28	92.03	93.57	93.50	93.21
1148	372	48	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	93.38	95.56	92.97	92.20	94.11	92.52	93.89	93.71	92.98	93.64	92.94	93.18	94.15	94.64	92.43	92.56	92.20	94.57	92.05	94.50	91.49	94.60	91.87	89.41	93.36
1151	372	48	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	93.22	93.55	96.29	93.81	95.24	93.19	94.56	94.51	93.71	94.03	93.07	90.28	96.01	94.81	93.84	95.88	93.81	94.27	91.42	92.05	91.28	93.07	94.25	92.91	92.39
1200	372	48	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	93.12	91.99	92.64	89.96	92.05	92.45	91.48	95.30	91.52	92.36	89.46	90.46	92.23	93.50	94.02	93.00	92.84	89.90	90.50	90.21	93.22	93.19	92.30	92.15	91.77
1203	372	48	0.1	16	0.00001	24	0.000000e+00	1.000000e-08	93.31	91.84	94.48	92.90	94.10	92.17	93.23	95.22	93.17	93.93	92.26	92.18	93.30	94.97	94.59	96.20	95.72	94.25	93.56	94.01	93.80	92.22	93.98	92.01	93.06
1212	372	48	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	93.47	93.89	93.02	93.69	94.65	92.55	94.66	92.39	94.12	95.05	93.61	93.11	94.09	96.36	90.96	92.87	92.32	95.49	92.90	90.50	91.72	93.99	92.91	91.59	92.65
1215	372	48	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	90.60	93.28	93.66	94.61	94.62	92.87	93.34	92.74	93.00	93.95	91.30	91.29	94.74	92.20	93.87	94.00	93.12	93.70	93.45	91.49	93.90	92.63	92.94	91.99	92.12
1264	372	48	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	91.19	92.33	93.61	92.87	94.49	93.06	92.33	92.80	93.26	95.28	94.09	90.32	94.94	94.10	92.91	94.09	93.52	93.22	91.80	90.81	92.44	90.89	92.77	90.80	89.40
1267	372	48	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	91.28	93.61	92.79	94.22	93.33	92.53	93.71	92.84	92.55	92.59	93.75	93.30	93.17	91.98	94.46	93.54	94.39	94.65	92.24	94.79	94.34	91.69	92.14	92.31	89.58
1276	372	48	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	93.66	93.66	93.94	92.90	92.52	94.58	94.42	92.46	93.40	94.52	92.22	92.68	92.67	94.56	93.38	94.28	96.05	94.68	93.36	94.03	91.53	92.14	94.29	92.22	91.56
1279	372	48	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	93.90	92.19	92.15	92.21	92.07	92.29	93.16	91.05	93.22	94.24	93.58	92.67	93.63	93.21	94.43	92.55	94.75	94.06	92.56	90.22	91.48	90.68	91.54	92.19	90.41
1328	372	48	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	93.20	95.10	94.31	89.79	94.68	94.08	90.67	92.42	94.07	94.68	93.33	91.78	94.55	96.77	92.92	93.10	95.04	95.56	93.15	91.98	91.20	93.88	94.27	92.29	93.15
1331	372	48	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	95.17	92.94	95.50	94.29	94.75	91.81	91.93	94.22	92.19	94.23	92.72	94.38	94.83	93.78	94.69	96.40	94.28	94.48	94.05	92.38	92.07	93.06	93.69	92.84	93.95
1340	372	48	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	94.17	94.52	94.37	95.15	93.62	92.10	94.79	92.55	94.39	95.00	93.61	94.49	93.88	95.06	91.38	95.31	94.99	93.54	93.84	93.57	91.41	91.85	94.13	93.24	92.30
1343	372	48	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	92.93	94.49	95.22	94.23	95.29	91.12	91.39	93.12	91.73	93.48	93.75	93.69	94.20	95.65	94.05	92.48	94.63	93.77	92.39	94.41	93.37	93.84	91.86	92.54	93.42
1392	372	48	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	92.98	91.78	94.22	92.35	93.50	91.95	92.74	95.09	93.32	92.53	93.03	91.45	93.50	94.65	94.51	95.02	94.04	92.59	90.84	90.69	92.77	92.77	94.01	92.45	92.27
1395	372	48	0.1	16	0.00001	36	0.000000e+00	1.000000e-08	94.29	94.30	95.18	95.65	95.29	92.63	93.17	95.50	92.93	95.58	93.50	85.32	94.65	95.34	95.77	94.69	95.95	95.55	92.84	93.04	93.08	95.33	92.52	94.10	91.94
1404	372	48	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	91.68	94.09	92.82	91.28	93.80	92.11	92.64	92.31	93.08	94.21	92.75	92.80	91.53	92.41	94.81	92.93	93.09	91.65	91.70	92.80	92.61	93.21	92.54	94.64	93.74
1407	372	48	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	91.38	94.04	93.95	95.93	94.36	91.23	93.83	90.61	95.15	94.01	93.04	93.50	95.15	93.75	94.00	94.33	93.84	93.39	94.01	91.62	91.99	90.60	93.01	94.60	93.20
1456	372	48	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	91.19	93.96	95.25	94.95	93.53	91.72	93.17	88.20	95.37	91.72	93.18	91.04	94.08	93.03	91.39	94.72	93.54	93.85	91.88	90.76	92.68	92.36	92.74	93.10	92.67
1459	372	48	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	94.47	94.72	93.29	93.30	92.34	92.66	91.40	93.72	92.17	94.28	91.57	92.08	93.59	93.93	92.00	95.55	95.11	92.01	93.69	91.93	93.11	92.69	93.49	92.88	91.48
1468	372	48	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	95.51	93.72	94.33	94.43	93.34	91.22	94.64	95.07	93.99	94.61	93.87	93.91	93.78	94.33	93.36	93.08	94.88	93.56	91.40	91.16	92.88	93.94	93.49	91.78	94.08
1471	372	48	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	93.04	94.32	95.52	91.98	95.25	92.57	94.78	93.67	93.58	95.08	92.89	91.94	91.43	95.09	92.80	94.19	93.09	94.03	92.11	90.11	93.27	94.02	92.81	92.60	93.10
1520	372	48	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	93.04	93.84	94.66	94.50	94.07	93.04	93.67	95.23	95.81	95.08	92.39	91.41	94.97	93.90	93.03	95.05	95.07	94.64	93.62	93.19	93.58	93.87	92.51	93.06	92.26
1523	372	48	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	94.35	95.04	95.11	95.30	96.06	93.08	95.21	92.99	95.05	95.90	93.02	92.23	95.29	95.80	94.87	95.78	94.41	93.82	91.70	93.52	92.77	92.20	92.98	91.74	92.81
1532	372	48	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	94.17	93.91	93.59	95.57	94.82	92.96	94.00	93.25	93.03	94.82	94.79	92.69	94.28	93.66	94.86	93.70	96.09	93.93	93.12	89.72	92.84	93.19	93.39	93.93	91.59
1535	372	48	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	94.87	94.80	94.28	95.28	94.42	93.59	92.80	95.34	93.48	93.19	93.12	92.23	93.32	95.04	94.55	95.23	93.06	93.55	93.65	92.18	94.48	92.46	93.70	91.94	93.71

Model performance on validation data (T-test)

Statistical Significance:

Using the global mean maximum row as the reference, we perform the right-tailed t-test to identify significant hyperparameters.

pvalues = []
pvalues_real = []

for _,index in enumerate(dt_valid_sub.index):
  _, p_value = stats.ttest_ind(max_row[:-1], dt_valid_sub.loc[index][:-1], alternative='greater')
  pvalues_real.append(p_value)
  pvalues.append((p_value >= 0.05)*1)

  

dt_valid_sub['pvalues'] = pvalues_real
dt_valid_sub['sig'] = pvalues


dt_valid_sub1 = dt_valid_sub.query('sig == 1')


max_indexx = dt_valid_sub1.iloc[:, :-3].mean().idxmax()

#print(sum(pvalues),max_indexx)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_zero	mAP_valid_zero_2	mAP_valid_zero_3	mAP_valid_zero_4	mAP_valid_zero_5	mAP_valid_zero_6	mAP_valid_zero_7	mAP_valid_zero_8	mAP_valid_zero_9	mAP_valid_zero_10	mAP_valid_zero_11	mAP_valid_zero_12	mAP_valid_zero_13	mAP_valid_zero_14	mAP_valid_zero_15	mAP_valid_zero_16	mAP_valid_zero_17	mAP_valid_zero_18	mAP_valid_zero_19	mAP_valid_zero_20	mAP_valid_zero_21	mAP_valid_zero_22	mAP_valid_zero_23	mAP_valid_zero_25	mAP_valid_zero_26
435	372	48	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	85.30	87.35	86.43	85.20	88.81	86.82	87.23	85.86	88.58	88.08	88.12	88.15	87.68	85.70	86.47	88.28	85.28	88.04	90.15	87.30	90.03	89.23	88.94	87.35	88.70
1468	372	48	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	87.24	86.62	86.06	85.13	87.90	88.96	86.40	84.54	88.66	86.36	88.23	87.04	87.20	87.12	87.26	88.77	88.15	86.60	91.25	87.59	89.03	88.63	89.19	88.59	89.80
1471	372	48	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	86.10	86.75	86.95	90.66	84.58	86.47	87.87	86.76	84.15	86.43	88.70	89.32	89.73	83.03	86.03	87.96	85.29	88.96	88.65	91.07	88.91	89.02	90.18	88.62	87.05
691	372	48	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	87.36	86.62	87.05	86.32	88.37	89.31	86.40	89.78	87.17	85.77	89.62	88.19	87.03	88.06	87.65	87.47	86.34	86.24	90.23	89.33	87.93	89.43	89.24	90.10	89.04
892	372	48	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	86.75	87.43	87.24	85.47	86.19	87.64	86.70	88.84	87.88	87.36	88.78	86.86	88.07	86.63	86.45	87.89	86.79	85.95	88.77	89.68	88.42	88.18	89.49	88.68	89.25
895	372	48	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	85.45	88.30	85.38	87.46	87.46	86.33	84.59	86.11	86.63	87.51	88.73	87.45	89.01	87.14	87.18	88.87	86.46	87.18	87.81	89.28	89.36	89.51	89.02	90.31	88.78
1084	372	48	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	88.63	88.33	87.01	86.57	89.05	88.56	87.57	87.38	89.00	86.30	87.32	89.44	87.40	88.07	86.77	89.25	85.38	87.85	87.80	89.60	89.31	88.06	87.94	89.04	90.95
307	372	48	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	86.73	87.99	86.54	86.40	86.67	89.27	87.74	86.81	89.71	86.17	87.91	88.32	87.14	87.46	86.59	88.63	86.55	88.45	88.79	89.51	89.08	87.59	88.33	89.97	85.85
316	372	48	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	87.97	87.19	86.32	86.34	87.51	86.80	85.44	85.73	87.22	87.14	87.76	87.18	88.11	86.65	87.70	86.70	86.74	87.49	91.85	89.82	89.47	88.73	89.06	91.07	91.68
319	372	48	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	86.46	86.67	86.34	87.15	85.56	88.09	89.13	86.37	88.11	85.97	89.27	88.36	87.43	87.66	86.40	85.95	87.84	85.50	88.21	88.75	87.69	89.97	87.71	88.93	88.75
508	372	48	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	88.05	86.54	87.80	86.96	85.97	87.13	87.32	86.47	86.02	86.03	90.38	88.90	89.56	86.16	86.25	87.55	85.76	86.38	88.53	88.91	88.51	86.70	87.90	88.25	90.45
1535	372	48	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	85.42	87.50	88.68	86.42	86.05	86.21	86.95	85.50	86.01	85.47	87.91	88.15	87.77	86.70	86.32	87.15	89.90	86.77	88.74	92.23	88.88	88.54	88.37	88.51	88.18
755	372	48	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	87.14	89.30	85.99	86.06	87.95	87.45	87.32	89.24	87.85	85.83	89.17	89.72	88.81	87.38	85.47	85.87	86.34	87.83	89.34	89.41	90.70	89.07	87.53	89.91	89.64
764	372	48	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	85.85	85.18	86.20	84.52	88.04	87.21	87.27	87.62	85.96	86.23	87.90	86.87	86.26	86.08	88.10	88.56	89.16	87.04	89.06	89.77	90.80	88.02	89.34	89.59	87.75
1151	372	48	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	86.06	86.47	86.56	88.02	84.68	86.86	87.41	89.53	86.65	87.54	87.58	88.94	88.64	86.36	86.17	87.12	85.26	86.82	87.66	87.84	89.05	89.52	89.82	89.19	88.31
176	372	48	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	87.68	86.69	43.72	84.53	83.73	86.66	85.94	87.61	86.84	87.96	87.53	88.18	86.48	85.56	87.26	86.63	86.36	87.18	87.39	86.94	89.36	89.13	84.66	88.02	87.17
188	372	48	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	84.51	88.84	86.31	85.26	86.95	87.67	86.64	88.40	87.31	88.42	88.58	87.57	87.37	83.06	84.12	89.56	85.67	87.02	88.59	90.20	89.16	88.69	90.84	88.18	88.10

Size of the All data:  (96, 28)
Size of the Sig data:  (17, 28)

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params1 = dt_valid.loc[dt_valid_sub1.index][['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
optimal_params = optimal_params1.reset_index()
optimal_params.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_1 = pd.merge(optimal_params, dt_test, on=['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg'])
test_data_1_sub = test_data_1[list(test_data_1.columns[test_data_1.columns.str.startswith('mAP_test')])]

Equivalent test data

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_zero	mAP_test_zero_2	mAP_test_zero_3	mAP_test_zero_4	mAP_test_zero_5	mAP_test_zero_6	mAP_test_zero_7	mAP_test_zero_8	mAP_test_zero_9	mAP_test_zero_10	mAP_test_zero_11	mAP_test_zero_12	mAP_test_zero_13	mAP_test_zero_14	mAP_test_zero_15	mAP_test_zero_16	mAP_test_zero_17	mAP_test_zero_18	mAP_test_zero_19	mAP_test_zero_20	mAP_test_zero_21	mAP_test_zero_22	mAP_test_zero_23	mAP_test_zero_25	mAP_test_zero_26
9	372	48	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	93.23	95.13	93.87	93.51	93.99	91.78	92.51	94.02	94.74	94.74	94.23	94.73	93.46	92.87	94.22	95.55	94.09	93.76	92.65	89.88	94.25	91.83	92.81	91.55	93.28
5	372	48	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	95.51	93.72	94.33	94.43	93.34	91.22	94.64	95.07	93.99	94.61	93.87	93.91	93.78	94.33	93.36	93.08	94.88	93.56	91.40	91.16	92.88	93.94	93.49	91.78	94.08
8	372	48	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	93.04	94.32	95.52	91.98	95.25	92.57	94.78	93.67	93.58	95.08	92.89	91.94	91.43	95.09	92.80	94.19	93.09	94.03	92.11	90.11	93.27	94.02	92.81	92.60	93.10
2	372	48	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	92.56	93.20	95.29	93.18	92.99	91.37	94.41	93.20	94.32	90.86	93.95	90.54	92.71	93.35	92.87	93.86	96.18	92.60	91.94	93.24	91.32	93.00	93.09	91.37	93.08
6	372	48	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	93.34	91.70	92.76	95.52	94.60	93.56	92.59	89.66	92.92	94.74	90.15	90.52	91.21	94.28	94.80	93.85	94.34	92.68	92.15	92.25	90.14	92.90	92.70	92.15	93.40
7	372	48	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	93.70	94.89	92.13	92.22	94.31	92.87	92.55	93.17	94.58	92.86	90.26	91.63	91.28	93.82	92.72	92.85	94.41	93.53	91.30	93.39	90.67	93.59	92.90	91.86	91.81
0	372	48	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	93.87	92.20	93.08	95.04	94.50	92.63	91.91	92.63	94.57	95.36	91.69	91.67	94.30	94.98	92.32	95.25	96.03	92.80	92.56	91.48	92.74	93.31	91.53	92.22	93.46
4	372	48	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	93.38	94.54	91.04	95.16	92.55	89.78	95.01	94.57	94.24	93.75	93.06	91.70	95.36	93.01	92.31	92.70	95.06	94.65	91.22	92.35	93.35	91.46	94.07	91.11	90.75
3	372	48	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	93.60	95.97	93.82	95.03	94.34	91.42	96.37	93.64	94.94	93.94	92.39	94.10	92.99	94.44	92.59	93.36	94.86	91.57	92.36	91.99	93.16	93.25	92.37	93.11	92.66
13	372	48	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	94.23	91.41	91.90	95.00	94.48	91.07	94.07	94.24	93.01	93.05	92.59	94.00	94.57	94.54	94.39	95.55	91.52	92.03	92.53	93.29	93.69	93.70	91.93	92.25	91.78
10	372	48	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	93.90	95.05	92.75	94.51	93.55	92.41	94.31	94.22	95.80	94.24	91.74	93.11	92.54	94.02	94.12	92.65	96.25	94.58	92.82	92.77	94.30	93.41	93.92	94.80	92.14
12	372	48	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	94.87	94.80	94.28	95.28	94.42	93.59	92.80	95.34	93.48	93.19	93.12	92.23	93.32	95.04	94.55	95.23	93.06	93.55	93.65	92.18	94.48	92.46	93.70	91.94	93.71
1	372	48	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	93.61	93.12	93.15	95.64	93.75	91.55	94.29	95.06	94.69	93.61	89.31	91.46	93.41	94.24	92.63	94.38	95.42	95.06	93.07	92.34	90.95	91.58	93.51	92.57	91.64
11	372	48	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	93.48	95.77	95.11	93.67	92.00	93.08	93.46	94.52	94.74	95.38	92.83	93.64	94.15	94.13	91.96	95.01	94.00	94.14	92.87	91.59	92.54	93.34	93.11	93.46	92.97
14	372	48	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	93.22	93.55	96.29	93.81	95.24	93.19	94.56	94.51	93.71	94.03	93.07	90.28	96.01	94.81	93.84	95.88	93.81	94.27	91.42	92.05	91.28	93.07	94.25	92.91	92.39
16	372	48	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	93.70	90.77	48.12	92.68	93.50	90.95	93.76	95.68	93.71	95.41	94.35	92.85	93.28	95.18	93.97	93.11	91.94	93.59	92.56	92.82	89.16	93.12	91.32	92.50	91.88
15	372	48	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	94.18	92.37	92.74	91.60	93.47	92.76	95.05	94.40	93.53	94.32	91.30	89.92	94.38	90.84	94.74	93.32	95.51	94.95	93.46	89.46	93.61	91.50	94.72	92.34	88.71

Size of the test data:  (17, 33)

Difference

diff = dt[['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
for i, cols in enumerate(zip(dt.columns,test_data_1.columns)):   
   diff[f'col_diff_{i}'] = (np.array(dt[cols[0]]) - np.array(test_data_1[cols[1]]))*-1

# Function to count negatives in a row
def count_negatives(row):
    return sum(1 for value in row if isinstance(value, (int, float)) and value < 0)

diff['negative_count'] = diff.apply(count_negatives, axis=1)
diff.sort_values(by =["negative_count"],inplace = True, ascending=False)

html_table = diff.to_html(index=True)

# Wrap in a scrollable div
scrollable_table = f"""
<div style="height: 400px; width: 100%; overflow-x: auto; overflow-y: auto;">
    {html_table}
</div>
"""
# Display the scrollable table
display(HTML(scrollable_table))

# 
# show(diff)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	col_diff_0	col_diff_1	col_diff_2	col_diff_3	col_diff_4	col_diff_5	col_diff_6	col_diff_7	col_diff_8	col_diff_9	col_diff_10	col_diff_11	col_diff_12	col_diff_13	col_diff_14	col_diff_15	col_diff_16	col_diff_17	col_diff_18	col_diff_19	col_diff_20	col_diff_21	col_diff_22	col_diff_23	col_diff_24	col_diff_25	col_diff_26	col_diff_27	col_diff_28	col_diff_29	col_diff_30	col_diff_31	col_diff_32	negative_count
188	372	48	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.67	3.53	6.43	6.34	6.52	5.09	8.41	6.00	6.22	5.90	2.72	2.35	7.01	7.78	10.62	3.76	9.84	7.93	4.87	-0.74	4.45	2.81	3.88	4.16	0.61	1
1471	372	48	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.94	7.57	8.57	1.32	10.67	6.10	6.91	6.91	9.43	8.65	4.19	2.62	1.70	12.06	6.77	6.23	7.80	5.07	3.46	-0.96	4.36	5.00	2.63	3.98	6.05	1
176	372	48	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.02	4.08	4.40	8.15	9.77	4.29	7.82	8.07	6.87	7.45	6.82	4.67	6.80	9.62	6.71	6.48	5.58	6.41	5.17	5.88	-0.20	3.99	6.66	4.48	4.71	1
1535	372	48	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.45	7.30	5.60	8.86	8.37	7.38	5.85	9.84	7.47	7.72	5.21	4.08	5.55	8.34	8.23	8.08	3.16	6.78	4.91	-0.05	5.60	3.92	5.33	3.43	5.53	1
319	372	48	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.77	4.74	5.56	7.85	8.92	2.98	4.94	7.87	4.90	7.08	3.32	5.64	7.14	6.88	7.99	9.60	3.68	6.53	4.32	4.54	6.00	3.73	4.22	3.32	3.03	0
1151	372	48	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.16	7.08	9.73	5.79	10.56	6.33	7.15	4.98	7.06	6.49	5.49	1.34	7.37	8.45	7.67	8.76	8.55	7.45	3.76	4.21	2.23	3.55	4.43	3.72	4.08	0
764	372	48	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.63	10.59	8.91	9.15	3.96	5.87	6.19	6.90	8.78	9.15	4.93	6.77	7.89	8.05	3.86	6.45	4.84	7.10	3.81	1.82	1.74	5.32	3.77	3.87	5.22	0
755	372	48	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.47	3.82	7.16	9.58	5.80	4.10	6.97	5.82	6.84	7.78	0.14	1.74	4.60	6.86	7.16	8.51	9.08	7.23	3.73	2.93	0.25	2.51	5.98	2.66	2.00	0
508	372	48	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.85	8.51	4.95	7.55	7.58	5.28	6.99	7.75	9.78	8.21	1.36	4.21	2.98	7.86	7.87	5.10	10.49	8.20	4.29	3.86	5.79	6.71	6.02	6.55	1.69	0
435	372	48	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.93	7.78	7.44	8.31	5.18	4.96	5.28	8.16	6.16	6.66	6.11	6.58	5.78	7.17	7.75	7.27	8.81	5.72	2.50	2.58	4.22	2.60	3.87	4.20	4.58	0
1468	372	48	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.27	7.10	8.27	9.30	5.44	2.26	8.24	10.53	5.33	8.25	5.64	6.87	6.58	7.21	6.10	4.31	6.73	6.96	0.15	3.57	3.85	5.31	4.30	3.19	4.28	0
307	372	48	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.65	6.55	4.50	8.76	5.88	0.51	7.27	7.76	4.53	7.58	5.15	3.38	8.22	5.55	5.72	4.07	8.51	6.20	2.43	2.84	4.27	3.87	5.74	1.14	4.90	0
1084	372	48	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.24	3.87	6.07	8.47	5.45	4.07	4.34	5.25	5.57	9.06	4.37	2.23	6.90	6.91	5.55	6.00	10.65	4.95	4.76	1.88	3.43	5.25	3.59	3.18	2.51	0
895	372	48	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.25	6.59	6.75	4.76	6.85	6.54	7.96	7.06	7.95	5.35	1.53	4.18	2.27	6.68	5.54	3.98	7.95	6.35	3.49	4.11	1.31	4.08	3.88	1.55	3.03	0
892	372	48	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.59	4.27	5.52	10.05	8.41	5.92	5.89	0.82	5.04	7.38	1.37	3.66	3.14	7.65	8.35	5.96	7.55	6.73	3.38	2.57	1.72	4.72	3.21	3.47	4.15	0
691	372	48	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.20	6.58	8.24	6.86	4.62	2.06	8.01	3.42	7.15	5.09	4.33	2.35	5.68	5.29	5.22	6.39	9.84	6.36	1.71	3.91	3.39	3.57	3.85	1.27	4.04	0
316	372	48	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.63	8.78	7.50	8.69	6.83	4.62	10.93	7.91	7.72	6.80	4.63	6.92	4.88	7.79	4.89	6.66	8.12	4.08	0.51	2.17	3.69	4.52	3.31	2.04	0.98	0

Model performance on valid data

Average & Std Deviation of the Significant rows:

	mean	std
mAP_valid_zero_20	89.248824	1.331033
mAP_valid_zero_21	89.158235	0.822984
mAP_valid_zero_25	89.077059	0.963151
mAP_valid_zero_19	88.989412	1.242017
mAP_valid_zero_26	88.791176	1.463207
mAP_valid_zero_22	88.707059	0.813348
mAP_valid_zero_23	88.680000	1.369457
mAP_valid_zero_11	88.440588	0.827235
mAP_valid_zero_12	88.155294	0.903515
mAP_valid_zero_13	87.864118	1.003463
mAP_valid_zero_16	87.777059	1.106519
mAP_valid_zero_6	87.496471	1.013829
mAP_valid_zero_2	87.280588	1.022585
mAP_valid_zero_9	87.279412	1.360168
mAP_valid_zero_8	87.208824	1.529497
mAP_valid_zero_18	87.135294	0.901132
mAP_valid_zero_7	86.936471	1.027855
mAP_valid_zero_5	86.792353	1.553715
mAP_valid_zero_10	86.739412	0.914525
mAP_valid_zero_17	86.662941	1.367021
mAP_valid_zero	86.629412	1.131009
mAP_valid_zero_15	86.599412	0.935965
mAP_valid_zero_14	86.401176	1.465375
mAP_valid_zero_4	86.380588	1.482584
mAP_valid_zero_3	84.151765	10.446424

Model performance on Test data

We then apply the hyperparameters to the test set and average the results.

test_data_1['average_map'] = test_data_1_sub.apply(np.mean, axis=1)

test_data_1.sort_values(by=['average_map'], ascending= False,inplace=True)

	mean	std
mAP_test_zero_17	94.379412	1.388711
mAP_test_zero_9	94.150000	0.754826
mAP_test_zero_16	94.107059	1.112659
mAP_test_zero_10	94.068824	1.156292
mAP_test_zero_14	94.057059	1.080166
mAP_test_zero_4	94.015294	1.308057
mAP_test_zero_8	93.976471	1.375654
mAP_test_zero_7	93.945294	1.168370
mAP_test_zero_5	93.898824	0.884194
mAP_test_zero	93.730588	0.693465
mAP_test_zero_2	93.677059	1.570620
mAP_test_zero_18	93.608824	1.001498
mAP_test_zero_15	93.422941	0.953302
mAP_test_zero_13	93.422353	1.347941
average_map	93.186141	0.609950
mAP_test_zero_23	93.072353	0.939777
mAP_test_zero_22	92.910588	0.843960
mAP_test_zero_21	92.458235	1.584169
mAP_test_zero_26	92.402353	1.293448
mAP_test_zero_11	92.400000	1.477303
mAP_test_zero_25	92.383529	0.872432
mAP_test_zero_19	92.357059	0.731187
mAP_test_zero_12	92.248824	1.487308
mAP_test_zero_6	92.105882	1.062615
mAP_test_zero_20	91.902941	1.178897
mAP_test_zero_3	90.951765	11.126060

Model performance on validation data (Mann-Whitney U Test)

Statistical Significance:

Using the global mean maximum row as the reference, we perform the right-tailed t-test to identify significant hyperparameters.

pvalues_mw = []
pvalues_real_mw = []

for _,index in enumerate(dt_valid_sub.index):

  _, p_value_mw = stats.mannwhitneyu(max_row[:-1], dt_valid_sub.loc[index][:-1], alternative='greater')#, method = 'asymptotic')
  pvalues_real_mw.append(p_value_mw)
  pvalues_mw.append((p_value_mw >= 0.05)*1)
  

dt_valid_sub_mw['pvalues_mw'] = pvalues_real_mw
dt_valid_sub_mw['sig_mw'] = pvalues_mw

dt_valid_sub_mw1 = dt_valid_sub_mw.query('sig_mw == 1')


max_indexx = dt_valid_sub_mw1.iloc[:, :-3].mean().idxmax()

#print(sum(pvalues),max_indexx)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_zero	mAP_valid_zero_2	mAP_valid_zero_3	mAP_valid_zero_4	mAP_valid_zero_5	mAP_valid_zero_6	mAP_valid_zero_7	mAP_valid_zero_8	mAP_valid_zero_9	mAP_valid_zero_10	mAP_valid_zero_11	mAP_valid_zero_12	mAP_valid_zero_13	mAP_valid_zero_14	mAP_valid_zero_15	mAP_valid_zero_16	mAP_valid_zero_17	mAP_valid_zero_18	mAP_valid_zero_19	mAP_valid_zero_20	mAP_valid_zero_21	mAP_valid_zero_22	mAP_valid_zero_23	mAP_valid_zero_25	mAP_valid_zero_26
1020	372	48	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	83.73	86.37	86.35	89.26	85.23	87.49	85.22	87.35	86.61	89.02	88.36	87.91	86.51	85.40	86.66	86.47	86.87	87.63	88.56	89.30	89.08	89.62	88.82	89.46	88.28
435	372	48	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	85.30	87.35	86.43	85.20	88.81	86.82	87.23	85.86	88.58	88.08	88.12	88.15	87.68	85.70	86.47	88.28	85.28	88.04	90.15	87.30	90.03	89.23	88.94	87.35	88.70
1468	372	48	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	87.24	86.62	86.06	85.13	87.90	88.96	86.40	84.54	88.66	86.36	88.23	87.04	87.20	87.12	87.26	88.77	88.15	86.60	91.25	87.59	89.03	88.63	89.19	88.59	89.80
1471	372	48	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	86.10	86.75	86.95	90.66	84.58	86.47	87.87	86.76	84.15	86.43	88.70	89.32	89.73	83.03	86.03	87.96	85.29	88.96	88.65	91.07	88.91	89.02	90.18	88.62	87.05
691	372	48	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	87.36	86.62	87.05	86.32	88.37	89.31	86.40	89.78	87.17	85.77	89.62	88.19	87.03	88.06	87.65	87.47	86.34	86.24	90.23	89.33	87.93	89.43	89.24	90.10	89.04
892	372	48	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	86.75	87.43	87.24	85.47	86.19	87.64	86.70	88.84	87.88	87.36	88.78	86.86	88.07	86.63	86.45	87.89	86.79	85.95	88.77	89.68	88.42	88.18	89.49	88.68	89.25
895	372	48	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	85.45	88.30	85.38	87.46	87.46	86.33	84.59	86.11	86.63	87.51	88.73	87.45	89.01	87.14	87.18	88.87	86.46	87.18	87.81	89.28	89.36	89.51	89.02	90.31	88.78
1084	372	48	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	88.63	88.33	87.01	86.57	89.05	88.56	87.57	87.38	89.00	86.30	87.32	89.44	87.40	88.07	86.77	89.25	85.38	87.85	87.80	89.60	89.31	88.06	87.94	89.04	90.95
307	372	48	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	86.73	87.99	86.54	86.40	86.67	89.27	87.74	86.81	89.71	86.17	87.91	88.32	87.14	87.46	86.59	88.63	86.55	88.45	88.79	89.51	89.08	87.59	88.33	89.97	85.85
316	372	48	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	87.97	87.19	86.32	86.34	87.51	86.80	85.44	85.73	87.22	87.14	87.76	87.18	88.11	86.65	87.70	86.70	86.74	87.49	91.85	89.82	89.47	88.73	89.06	91.07	91.68
755	372	48	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	87.14	89.30	85.99	86.06	87.95	87.45	87.32	89.24	87.85	85.83	89.17	89.72	88.81	87.38	85.47	85.87	86.34	87.83	89.34	89.41	90.70	89.07	87.53	89.91	89.64
764	372	48	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	85.85	85.18	86.20	84.52	88.04	87.21	87.27	87.62	85.96	86.23	87.90	86.87	86.26	86.08	88.10	88.56	89.16	87.04	89.06	89.77	90.80	88.02	89.34	89.59	87.75
188	372	48	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	84.51	88.84	86.31	85.26	86.95	87.67	86.64	88.40	87.31	88.42	88.58	87.57	87.37	83.06	84.12	89.56	85.67	87.02	88.59	90.20	89.16	88.69	90.84	88.18	88.10

Size of the All data:  (96, 28)
Size of the Sig data:  (13, 33)

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params_mw1 = dt_valid.loc[dt_valid_sub_mw1.index][['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
optimal_params_mw = optimal_params_mw1.reset_index()
optimal_params_mw.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_mw1 = pd.merge(optimal_params_mw, dt_test, on=['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg'])
test_data_1_sub_mw = test_data_mw1[list(test_data_mw1.columns[test_data_mw1.columns.str.startswith('mAP_test')])]

Equivalent test data

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_zero	mAP_test_zero_2	mAP_test_zero_3	mAP_test_zero_4	mAP_test_zero_5	mAP_test_zero_6	mAP_test_zero_7	mAP_test_zero_8	mAP_test_zero_9	mAP_test_zero_10	mAP_test_zero_11	mAP_test_zero_12	mAP_test_zero_13	mAP_test_zero_14	mAP_test_zero_15	mAP_test_zero_16	mAP_test_zero_17	mAP_test_zero_18	mAP_test_zero_19	mAP_test_zero_20	mAP_test_zero_21	mAP_test_zero_22	mAP_test_zero_23	mAP_test_zero_25	mAP_test_zero_26
12	372	48	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	93.96	95.67	93.92	94.53	93.34	92.91	95.05	92.34	92.96	95.16	92.88	91.60	92.47	93.61	93.45	93.16	93.49	92.94	93.48	92.31	90.62	92.68	93.93	92.42	92.49
9	372	48	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	93.23	95.13	93.87	93.51	93.99	91.78	92.51	94.02	94.74	94.74	94.23	94.73	93.46	92.87	94.22	95.55	94.09	93.76	92.65	89.88	94.25	91.83	92.81	91.55	93.28
5	372	48	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	95.51	93.72	94.33	94.43	93.34	91.22	94.64	95.07	93.99	94.61	93.87	93.91	93.78	94.33	93.36	93.08	94.88	93.56	91.40	91.16	92.88	93.94	93.49	91.78	94.08
8	372	48	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	93.04	94.32	95.52	91.98	95.25	92.57	94.78	93.67	93.58	95.08	92.89	91.94	91.43	95.09	92.80	94.19	93.09	94.03	92.11	90.11	93.27	94.02	92.81	92.60	93.10
2	372	48	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	92.56	93.20	95.29	93.18	92.99	91.37	94.41	93.20	94.32	90.86	93.95	90.54	92.71	93.35	92.87	93.86	96.18	92.60	91.94	93.24	91.32	93.00	93.09	91.37	93.08
6	372	48	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	93.34	91.70	92.76	95.52	94.60	93.56	92.59	89.66	92.92	94.74	90.15	90.52	91.21	94.28	94.80	93.85	94.34	92.68	92.15	92.25	90.14	92.90	92.70	92.15	93.40
7	372	48	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	93.70	94.89	92.13	92.22	94.31	92.87	92.55	93.17	94.58	92.86	90.26	91.63	91.28	93.82	92.72	92.85	94.41	93.53	91.30	93.39	90.67	93.59	92.90	91.86	91.81
0	372	48	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	93.87	92.20	93.08	95.04	94.50	92.63	91.91	92.63	94.57	95.36	91.69	91.67	94.30	94.98	92.32	95.25	96.03	92.80	92.56	91.48	92.74	93.31	91.53	92.22	93.46
4	372	48	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	93.38	94.54	91.04	95.16	92.55	89.78	95.01	94.57	94.24	93.75	93.06	91.70	95.36	93.01	92.31	92.70	95.06	94.65	91.22	92.35	93.35	91.46	94.07	91.11	90.75
3	372	48	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	93.60	95.97	93.82	95.03	94.34	91.42	96.37	93.64	94.94	93.94	92.39	94.10	92.99	94.44	92.59	93.36	94.86	91.57	92.36	91.99	93.16	93.25	92.37	93.11	92.66
1	372	48	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	93.61	93.12	93.15	95.64	93.75	91.55	94.29	95.06	94.69	93.61	89.31	91.46	93.41	94.24	92.63	94.38	95.42	95.06	93.07	92.34	90.95	91.58	93.51	92.57	91.64
10	372	48	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	93.48	95.77	95.11	93.67	92.00	93.08	93.46	94.52	94.74	95.38	92.83	93.64	94.15	94.13	91.96	95.01	94.00	94.14	92.87	91.59	92.54	93.34	93.11	93.46	92.97
11	372	48	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	94.18	92.37	92.74	91.60	93.47	92.76	95.05	94.40	93.53	94.32	91.30	89.92	94.38	90.84	94.74	93.32	95.51	94.95	93.46	89.46	93.61	91.50	94.72	92.34	88.71

Size of the test data:  (13, 33)

Difference

diff_mw = dt_mw[['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
for i, cols in enumerate(zip(dt_mw.columns,test_data_mw1.columns)):   
   diff_mw[f'col_diff_{i}'] = (np.array(dt_mw[cols[0]]) - np.array(test_data_mw1[cols[1]]))*-1

# Function to count negatives in a row
def count_negatives(row):
    return sum(1 for value in row if isinstance(value, (int, float)) and value < 0)

diff_mw['negative_count'] = diff_mw.apply(count_negatives, axis=1)

diff_mw.sort_values(by =["negative_count"],inplace = True, ascending=False)


html_table = diff_mw.to_html(index=True)

# Wrap in a scrollable div
scrollable_table = f"""
<div style="height: 400px; width: 100%; overflow-x: auto; overflow-y: auto;">
    {html_table}
</div>
"""
# Display the scrollable table
display(HTML(scrollable_table))

# show(diff)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	col_diff_0	col_diff_1	col_diff_2	col_diff_3	col_diff_4	col_diff_5	col_diff_6	col_diff_7	col_diff_8	col_diff_9	col_diff_10	col_diff_11	col_diff_12	col_diff_13	col_diff_14	col_diff_15	col_diff_16	col_diff_17	col_diff_18	col_diff_19	col_diff_20	col_diff_21	col_diff_22	col_diff_23	col_diff_24	col_diff_25	col_diff_26	col_diff_27	col_diff_28	col_diff_29	col_diff_30	col_diff_31	col_diff_32	negative_count
1471	372	48	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.94	7.57	8.57	1.32	10.67	6.10	6.91	6.91	9.43	8.65	4.19	2.62	1.70	12.06	6.77	6.23	7.80	5.07	3.46	-0.96	4.36	5.00	2.63	3.98	6.05	1
188	372	48	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.67	3.53	6.43	6.34	6.52	5.09	8.41	6.00	6.22	5.90	2.72	2.35	7.01	7.78	10.62	3.76	9.84	7.93	4.87	-0.74	4.45	2.81	3.88	4.16	0.61	1
1020	372	48	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	10.23	9.30	7.57	5.27	8.11	5.42	9.83	4.99	6.35	6.14	4.52	3.69	5.96	8.21	6.79	6.69	6.62	5.31	4.92	3.01	1.54	3.06	5.11	2.96	4.21	0
435	372	48	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.93	7.78	7.44	8.31	5.18	4.96	5.28	8.16	6.16	6.66	6.11	6.58	5.78	7.17	7.75	7.27	8.81	5.72	2.50	2.58	4.22	2.60	3.87	4.20	4.58	0
1468	372	48	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.27	7.10	8.27	9.30	5.44	2.26	8.24	10.53	5.33	8.25	5.64	6.87	6.58	7.21	6.10	4.31	6.73	6.96	0.15	3.57	3.85	5.31	4.30	3.19	4.28	0
691	372	48	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.20	6.58	8.24	6.86	4.62	2.06	8.01	3.42	7.15	5.09	4.33	2.35	5.68	5.29	5.22	6.39	9.84	6.36	1.71	3.91	3.39	3.57	3.85	1.27	4.04	0
892	372	48	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.59	4.27	5.52	10.05	8.41	5.92	5.89	0.82	5.04	7.38	1.37	3.66	3.14	7.65	8.35	5.96	7.55	6.73	3.38	2.57	1.72	4.72	3.21	3.47	4.15	0
895	372	48	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.25	6.59	6.75	4.76	6.85	6.54	7.96	7.06	7.95	5.35	1.53	4.18	2.27	6.68	5.54	3.98	7.95	6.35	3.49	4.11	1.31	4.08	3.88	1.55	3.03	0
1084	372	48	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.24	3.87	6.07	8.47	5.45	4.07	4.34	5.25	5.57	9.06	4.37	2.23	6.90	6.91	5.55	6.00	10.65	4.95	4.76	1.88	3.43	5.25	3.59	3.18	2.51	0
307	372	48	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.65	6.55	4.50	8.76	5.88	0.51	7.27	7.76	4.53	7.58	5.15	3.38	8.22	5.55	5.72	4.07	8.51	6.20	2.43	2.84	4.27	3.87	5.74	1.14	4.90	0
316	372	48	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.63	8.78	7.50	8.69	6.83	4.62	10.93	7.91	7.72	6.80	4.63	6.92	4.88	7.79	4.89	6.66	8.12	4.08	0.51	2.17	3.69	4.52	3.31	2.04	0.98	0
755	372	48	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.47	3.82	7.16	9.58	5.80	4.10	6.97	5.82	6.84	7.78	0.14	1.74	4.60	6.86	7.16	8.51	9.08	7.23	3.73	2.93	0.25	2.51	5.98	2.66	2.00	0
764	372	48	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.63	10.59	8.91	9.15	3.96	5.87	6.19	6.90	8.78	9.15	4.93	6.77	7.89	8.05	3.86	6.45	4.84	7.10	3.81	1.82	1.74	5.32	3.77	3.87	5.22	0

Model performance on valid data

Average & Std Deviation of the Significant rows:

	mean	std
mAP_valid_zero_20	89.373846	0.982315
mAP_valid_zero_21	89.329231	0.804005
mAP_valid_zero_25	89.297692	1.008259
mAP_valid_zero_19	89.296154	1.239513
mAP_valid_zero_23	89.070769	0.862926
mAP_valid_zero_26	88.836154	1.541209
mAP_valid_zero_22	88.752308	0.639038
mAP_valid_zero_11	88.398462	0.621287
mAP_valid_zero_16	88.021538	1.117705
mAP_valid_zero_12	88.001538	0.983250
mAP_valid_zero_13	87.716923	1.002990
mAP_valid_zero_6	87.690769	1.029097
mAP_valid_zero_9	87.440769	1.450112
mAP_valid_zero_18	87.406154	0.856549
mAP_valid_zero_2	87.405385	1.132193
mAP_valid_zero_5	87.285385	1.332895
mAP_valid_zero_8	87.263077	1.519130
mAP_valid_zero_10	86.970769	1.042812
mAP_valid_zero_15	86.650000	1.045873
mAP_valid_zero_7	86.645385	1.023577
mAP_valid_zero_17	86.540000	1.118295
mAP_valid_zero_4	86.511538	1.736744
mAP_valid_zero_3	86.448462	0.514277
mAP_valid_zero	86.366154	1.385494
mAP_valid_zero_14	86.290769	1.652263

Model performance on Test data

We then apply the hyperparameters to the test set and average the results.

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params = dt_valid.loc[dt_valid_sub_mw1.index].iloc[:, :6].reset_index()
optimal_params.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_1 = pd.merge(optimal_params, dt_test, how='left')
test_data_1_sub = test_data_1[list(test_data_1.columns[test_data_1.columns.str.startswith('mAP_test')])]
test_data_1['average_map'] = test_data_1_sub.apply(np.mean, axis=1)

test_data_1.sort_values(by=['average_map'], ascending= False,inplace=True)

	mean	std
mAP_test_zero_17	94.212885	1.183352
mAP_test_zero_4	94.208269	1.196104
mAP_test_zero_14	94.156923	0.951938
mAP_test_zero_16	94.035385	1.115107
mAP_test_zero_5	93.923462	1.152222
mAP_test_zero_7	93.747115	1.278725
mAP_test_zero_9	93.680192	1.088319
mAP_test_zero_13	93.643269	1.195306
mAP_test_zero_18	93.636154	1.294183
mAP_test_zero_8	93.590962	1.803475
mAP_test_zero	93.355769	1.739785
mAP_test_zero_15	93.309615	1.291480
mAP_test_zero_10	93.298077	6.501133
average_map	92.954485	0.982671
mAP_test_zero_23	92.926923	0.765326
mAP_test_zero_3	92.749038	6.429037
mAP_test_zero_21	92.640192	1.292275
mAP_test_zero_12	92.471538	1.342197
mAP_test_zero_11	92.450962	1.327039
mAP_test_zero_26	92.386154	1.157386
mAP_test_zero_2	92.102692	8.981174
mAP_test_zero_6	91.862500	1.706873
mAP_test_zero_19	91.680000	6.231033
mAP_test_zero_25	91.590385	6.184148
mAP_test_zero_22	91.165000	8.744420
mAP_test_zero_20	91.038654	6.176708

Summary using radar plot

res1_valid['id'] = res1_valid.index.to_series().apply(extract_number)
res1_test['id'] = res1_test.index.to_series().apply(extract_number)



res_comb = pd.concat([res1_valid,res1_test])
index_series = res_comb.index.to_series()

res_comb['type'] = np.where(
    index_series.str.contains('valid', case=False), 'valid',
    np.where(index_series.str.contains('test', case=False), 'test', 'unknown')
)
res_comb = res_comb.query("type !='unknown'")

res_comb =  res_comb.reset_index(drop=True)


res1_test = res1_test.sort_values(by=['id']).reset_index().query("index !='average_map'")

data_range1 = np.array(list(res1_valid['mean']) + list(res1_test['mean']))

categories = [str(i) for i in range(1,26)]

fig = go.Figure()

# Valid
fig.add_trace(go.Scatterpolar(
       r=list(res1_test['mean']),
      theta=categories,
      #fill='toself',
      name='Test'
))

# Test
fig.add_trace(go.Scatterpolar(
      r=list(res1_valid['mean']),
      theta=categories,
      #fill='toself',
      name='Valid'
))

# Customization of chart
fig.update_layout(
  title={
      'text': "T-Test",
        'xanchor': 'center',
      'yanchor': 'top'
  },
  polar=dict(
    radialaxis=dict(
      visible=True,
      range=[round(data_range1.min()), round(data_range1.max()) + 1]
    )),
  showlegend=True
)

fig.show()




##############


res2_valid['id'] = res2_valid.index.to_series().apply(extract_number)
res2_test['id'] = res2_test.index.to_series().apply(extract_number)


res_comb = pd.concat([res2_valid,res2_test])
index_series = res_comb.index.to_series()

res_comb['type'] = np.where(
    index_series.str.contains('valid', case=False), 'valid',
    np.where(index_series.str.contains('test', case=False), 'test', 'unknown')
)
res_comb = res_comb.query("type !='unknown'")

res_comb =  res_comb.reset_index(drop=True)


res2_test = res2_test.sort_values(by=['id']).reset_index().query("index !='average_map'")

data_range2 = np.array(list(res2_valid['mean']) + list(res2_test['mean']))

categories = [str(i) for i in range(1,26)]
fig = go.Figure()

# Valid
fig.add_trace(go.Scatterpolar(
       r=list(res2_test['mean']),
      theta=categories,
      #fill='toself',
      name='Test'
))

# Test
fig.add_trace(go.Scatterpolar(
      r=list(res2_valid['mean']),
      theta=categories,
      #fill='toself',
      name='Valid'
))

# Customization of chart
fig.update_layout(
  title={
      'text': "MU-Test",
         'xanchor': 'center',
      'yanchor': 'top'
  },
  polar=dict(
    radialaxis=dict(
      visible=True,
      range=[round(data_range2.min()), round(data_range2.max()) + 1]
    )),
  showlegend=True
)

fig.show()

best_valid_bit_size_48 = round(res1_valid['mean'][0])
id_best_valid = res1_valid['id'][0]
best_test_bit_size_48 = list(round(res1_test.query('id == @id_best_valid')['mean'],2))[0]

best_valid_bit_size_48_mw = round(res2_valid['mean'][0])
id_best_valid = res2_valid['id'][0]
best_test_bit_size_48_mw = list(round(res2_test.query('id == @id_best_valid')['mean'],2))[0]

Bit size 16

The results in this presentation are from two experimental designs:

Experiment 1 Results

The thresholding is based on fixed values between -1 and 1 on a step size of 0.1.

Validation Data mAP Results Preview:

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_abs_values	mAP_valid_abs_values_2	mAP_valid_abs_values_3	mAP_valid_abs_values_4	mAP_valid_abs_values_5	mAP_valid_abs_values_6	mAP_valid_abs_values_7	mAP_valid_abs_values_8	mAP_valid_abs_values_9	mAP_valid_abs_values_10	mAP_valid_abs_values_11	mAP_valid_abs_values_12	mAP_valid_abs_values_13	mAP_valid_abs_values_14	mAP_valid_abs_values_15	mAP_valid_abs_values_16	mAP_valid_abs_values_17	mAP_valid_abs_values_18	mAP_valid_abs_values_19	mAP_valid_abs_values_20	mAP_valid_abs_values_21	mAP_valid_abs_values_22	mAP_valid_abs_values_23	mAP_valid_abs_values_25	mAP_valid_abs_values_26
16	372	16	0.1	16	0.00100	24	0.000000e+00	0.000000e+00	83.31	86.95	67.78	86.19	86.38	86.91	82.53	81.63	80.29	87.29	86.84	88.19	88.13	60.76	82.11	82.38	85.58	85.32	79.59	86.64	87.66	79.98	86.54	66.11	85.88
19	372	16	0.1	16	0.00100	24	0.000000e+00	1.000000e-08	83.32	87.74	84.84	73.32	82.80	84.29	82.88	86.42	86.24	85.61	85.19	86.81	84.30	85.63	87.14	87.08	88.34	88.56	88.81	88.48	87.70	85.39	84.07	88.04	86.52
28	372	16	0.1	16	0.00100	24	1.000000e-08	0.000000e+00	83.00	86.13	84.21	89.08	85.19	84.60	85.32	86.56	89.47	83.73	86.45	86.68	86.11	89.45	85.21	83.17	87.06	73.42	88.71	81.45	89.47	88.45	86.41	89.64	87.75
31	372	16	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	83.12	83.99	84.14	87.06	84.22	84.83	83.96	81.23	85.18	88.01	88.70	89.54	84.93	88.06	88.10	83.02	43.72	86.92	89.01	84.25	84.89	86.21	86.38	87.58	86.19
80	372	16	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	84.83	88.42	84.03	84.61	87.24	84.92	84.39	87.86	87.97	85.69	90.03	86.42	87.25	43.72	90.43	86.88	86.93	88.62	86.36	87.33	89.49	88.90	87.10	89.67	86.19
83	372	16	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	86.14	85.10	87.21	86.01	86.26	86.64	87.30	86.68	88.58	85.51	89.28	85.47	89.02	88.46	89.49	88.29	88.45	89.81	90.12	88.13	89.61	88.90	89.81	89.34	89.20
92	372	16	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	85.66	55.61	86.21	87.07	82.88	86.35	83.86	88.87	90.69	84.39	87.88	92.49	87.78	85.49	90.85	87.91	89.03	90.32	88.94	88.65	87.86	88.46	89.81	89.91	89.77
95	372	16	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	84.67	85.95	84.21	89.47	85.77	90.28	87.24	88.50	86.75	87.52	84.52	53.51	85.93	86.60	89.88	87.80	86.05	86.87	88.55	89.62	87.66	88.76	89.07	87.97	91.66
144	372	16	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	84.32	84.79	86.49	86.62	85.24	88.61	85.96	88.21	87.19	87.61	88.45	88.65	86.93	87.94	88.30	55.46	43.72	88.27	87.74	89.04	88.92	88.51	89.34	91.75	89.32
147	372	16	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	86.38	85.67	83.84	89.78	86.49	84.83	86.89	87.94	85.84	88.76	87.55	91.69	87.17	87.06	43.72	86.51	43.72	89.07	89.58	90.44	88.87	88.90	88.62	90.41	88.78
156	372	16	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	84.03	85.49	84.62	84.11	89.00	89.08	85.88	84.64	85.14	85.31	91.59	89.31	86.01	86.88	87.95	88.05	87.15	87.36	88.27	87.60	89.02	89.00	89.47	90.10	90.79
159	372	16	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	86.63	84.66	84.54	83.76	86.59	85.53	86.50	86.99	87.35	88.42	87.92	90.59	88.28	86.22	87.39	88.12	87.10	88.33	86.72	93.02	89.38	90.40	87.25	90.93	90.07
208	372	16	0.1	16	0.00100	36	0.000000e+00	0.000000e+00	87.20	88.25	63.66	63.93	62.85	59.49	57.47	62.04	62.62	64.34	58.63	59.23	89.62	60.53	61.34	58.66	61.74	62.89	90.32	59.32	60.43	61.18	60.42	62.23	60.49
211	372	16	0.1	16	0.00100	36	0.000000e+00	1.000000e-08	88.65	62.01	56.70	64.45	43.72	59.49	88.10	59.99	56.15	62.36	58.03	58.90	43.72	58.61	58.92	90.43	61.63	59.44	91.53	57.02	61.21	43.72	58.93	59.07	57.51
220	372	16	0.1	16	0.00100	36	1.000000e-08	0.000000e+00	56.27	63.90	87.95	63.49	65.26	56.97	86.29	62.26	85.19	63.81	61.84	58.84	88.94	62.77	63.19	87.23	57.24	88.84	60.44	56.89	67.26	67.33	56.16	62.61	63.69
223	372	16	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	86.65	55.70	87.57	56.78	63.28	90.66	89.44	61.60	59.97	62.12	91.70	92.06	89.04	59.46	61.42	89.82	61.28	89.54	90.65	56.57	60.08	60.03	60.45	59.92	57.87
272	372	16	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	89.69	60.96	61.84	60.62	61.37	90.68	55.70	56.01	84.84	60.60	92.19	93.93	89.38	92.14	59.74	91.96	60.39	90.10	92.23	58.76	58.97	57.83	90.23	58.24	58.94
275	372	16	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	89.81	59.88	60.17	60.38	90.59	91.19	85.34	91.72	59.53	85.35	89.17	59.01	91.18	60.01	59.70	91.67	55.78	91.43	92.20	92.98	60.29	59.83	58.48	59.29	57.95
284	372	16	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	88.12	56.48	90.11	62.18	62.70	90.43	89.46	59.56	91.34	61.07	91.97	92.70	89.76	90.04	61.86	91.19	55.31	91.03	91.43	92.35	59.89	58.29	89.92	59.07	58.83
287	372	16	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	88.39	90.65	88.85	61.19	88.63	92.22	89.61	59.68	91.54	91.49	91.05	58.79	90.50	91.45	92.65	82.95	59.15	90.62	58.93	91.19	56.89	58.14	59.40	60.00	57.57
336	372	16	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	87.64	61.30	87.87	60.32	90.47	91.06	89.66	59.74	90.93	59.59	92.06	89.31	87.80	58.86	59.21	89.18	58.78	91.43	89.09	91.96	58.79	58.44	59.20	59.65	57.92
339	372	16	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	87.41	90.69	86.65	60.40	88.16	92.18	85.83	59.98	90.40	89.16	89.79	58.52	84.75	58.06	60.38	90.91	91.71	90.19	57.70	58.87	58.35	56.58	90.42	56.61	57.64
348	372	16	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	89.68	61.30	88.66	60.53	89.72	93.22	89.91	60.42	88.25	90.94	58.35	93.18	87.08	91.68	90.29	91.42	59.61	93.22	89.16	92.17	55.85	57.59	92.46	57.11	57.41
351	372	16	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	87.64	59.90	87.64	61.82	89.67	89.14	88.40	89.41	89.93	89.90	91.80	89.03	91.65	87.67	43.72	86.68	90.92	90.42	57.16	89.15	88.62	56.83	58.84	59.35	60.03
400	372	16	0.1	16	0.00100	48	0.000000e+00	0.000000e+00	88.54	62.39	56.40	63.56	63.33	56.76	63.43	62.63	62.56	61.88	59.08	59.72	57.54	62.68	43.72	89.15	62.88	61.58	61.19	60.57	61.53	62.76	62.94	63.04	64.98
403	372	16	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	57.43	60.88	55.84	56.18	59.46	57.22	57.87	61.17	56.08	88.05	90.59	56.43	88.86	59.59	60.39	91.11	60.16	60.00	56.84	56.79	62.50	59.11	60.19	43.72	60.41
412	372	16	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	43.72	87.83	62.31	64.05	63.74	56.23	87.76	61.25	57.21	63.30	89.97	59.61	89.13	61.85	60.91	62.26	63.32	62.42	91.44	59.35	61.88	58.68	63.36	59.29	62.87
415	372	16	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	54.68	54.53	57.72	57.97	57.47	89.79	90.47	59.15	88.48	57.01	53.98	91.77	91.73	57.10	59.09	55.37	89.89	91.35	91.77	54.06	54.96	57.70	51.88	58.47	57.64
464	372	16	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	88.70	60.70	55.77	43.72	60.70	56.03	87.67	59.31	90.62	63.19	57.35	91.96	90.42	91.85	59.98	92.11	60.95	59.37	58.80	58.79	57.94	60.07	58.90	58.11	59.91
467	372	16	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	87.78	62.47	88.83	61.67	61.56	90.91	90.45	61.07	60.43	90.44	91.16	59.53	91.67	60.58	60.47	91.68	61.12	90.72	92.49	86.92	58.47	61.03	53.40	61.78	60.90
476	372	16	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	87.91	62.80	89.45	89.06	89.69	56.30	88.28	61.77	56.21	90.93	88.21	59.78	90.91	90.34	60.52	91.20	59.98	88.94	91.16	58.67	57.62	61.05	88.60	58.89	58.43
479	372	16	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	87.95	61.87	90.22	61.29	88.97	92.04	91.37	59.78	56.52	90.71	92.44	57.58	90.51	58.92	89.45	91.93	91.73	83.90	92.47	90.16	88.72	55.90	89.65	57.97	56.25
528	372	16	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	90.17	90.92	59.26	90.49	88.41	92.18	89.90	90.13	90.71	89.42	92.47	56.56	87.56	60.02	58.79	90.37	59.55	92.83	57.08	58.76	58.85	56.76	56.88	58.04	56.86
531	372	16	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	88.23	88.67	88.41	90.09	89.81	89.27	90.29	59.27	88.25	89.66	89.78	58.16	90.36	90.97	60.61	90.91	59.27	91.15	92.44	90.37	89.38	57.79	58.88	57.38	58.39
540	372	16	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	87.16	61.01	88.87	60.54	88.67	56.81	88.89	58.66	87.67	58.46	88.60	56.72	89.51	89.05	60.25	88.09	59.83	90.92	90.28	59.16	54.51	56.15	56.95	57.13	56.86
543	372	16	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	88.02	60.30	61.16	87.78	88.67	90.23	87.84	57.89	59.40	61.42	57.91	93.05	90.65	57.54	60.23	91.68	59.80	91.42	90.15	87.97	58.27	56.80	56.35	58.82	56.39
592	372	16	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	84.27	81.09	86.09	81.65	85.54	84.78	83.15	82.95	83.82	85.06	87.62	87.47	83.61	88.98	85.86	79.13	87.78	81.79	86.23	88.53	83.26	85.10	79.83	82.99	86.21
595	372	16	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	82.91	84.67	83.72	85.49	87.59	82.54	85.47	87.01	83.38	86.98	87.04	85.99	85.16	87.68	88.10	85.85	89.45	87.30	87.84	65.67	86.59	75.95	88.21	88.73	87.62
604	372	16	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	85.43	83.79	86.74	83.59	85.85	83.42	80.81	86.78	85.61	85.01	61.57	86.19	85.66	87.62	89.35	85.44	87.95	86.99	88.24	87.81	87.03	86.52	87.12	87.57	88.44
607	372	16	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	67.50	69.28	88.19	88.72	88.74	85.57	83.91	90.79	90.45	90.32	88.88	65.01	87.98	90.31	84.46	83.94	87.54	86.43	82.93	86.99	90.74	88.20	88.15	89.14	88.23
656	372	16	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	60.13	88.09	85.83	85.06	86.28	86.65	84.99	56.50	85.09	86.54	86.77	86.30	86.40	86.48	88.47	88.77	86.84	87.85	89.70	89.97	88.29	89.14	89.27	90.34	88.44
659	372	16	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	84.52	86.55	85.38	84.63	82.42	85.52	83.98	86.72	85.40	90.50	86.34	89.77	88.63	91.02	86.47	87.33	86.67	86.96	89.29	91.82	88.34	89.81	88.81	91.15	88.36
668	372	16	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	85.59	86.09	86.06	86.18	84.20	87.39	83.80	89.21	86.84	86.47	90.30	86.26	86.55	86.31	88.75	88.86	89.61	86.83	90.76	88.24	89.45	89.32	89.08	91.03	88.43
671	372	16	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	83.26	86.23	86.92	86.67	87.02	88.96	83.94	87.96	85.68	87.44	86.99	88.83	86.02	88.41	87.44	88.89	86.44	87.11	90.42	87.46	87.76	89.84	90.64	88.32	88.15
720	372	16	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	84.13	86.57	83.83	87.60	84.34	86.64	86.54	85.90	88.58	84.97	87.53	88.43	87.43	86.89	87.65	86.53	90.35	88.53	90.74	91.37	94.45	88.25	88.48	88.75	86.88
723	372	16	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	87.23	89.30	86.63	87.65	86.57	91.51	84.03	89.13	89.00	87.01	90.99	87.75	87.29	88.16	89.22	86.77	87.58	88.77	88.82	93.96	91.15	89.34	87.55	56.13	90.02
732	372	16	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	82.88	85.26	85.21	43.72	84.03	88.06	84.79	88.69	85.21	85.28	90.98	87.30	88.21	88.46	85.99	88.31	86.74	86.63	85.75	87.24	88.77	89.08	88.29	89.33	89.03
735	372	16	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	84.30	85.92	84.63	84.90	82.97	88.29	84.36	87.53	86.64	87.78	90.68	87.85	89.74	86.51	88.48	85.88	90.90	85.08	89.24	89.57	90.12	89.68	88.58	90.50	91.32
784	372	16	0.1	16	0.00010	36	0.000000e+00	0.000000e+00	57.93	63.47	63.25	63.39	63.58	60.56	86.48	63.60	57.25	63.50	59.93	56.08	88.42	62.11	63.10	61.22	62.28	62.17	58.82	60.20	64.55	64.11	65.23	62.92	61.72
787	372	16	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	87.93	61.74	62.84	60.99	61.57	90.04	87.83	59.68	90.33	61.96	85.60	59.67	83.26	62.19	62.23	90.17	59.94	62.92	59.01	91.50	62.18	59.70	61.82	61.96	58.91
796	372	16	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	87.43	64.57	56.88	64.58	65.34	90.02	88.11	63.22	88.39	64.82	90.92	58.61	90.91	63.35	61.89	62.48	64.16	89.59	59.31	59.77	65.40	66.61	60.65	62.51	60.37
799	372	16	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	88.73	55.47	59.85	61.14	60.20	91.33	87.83	60.31	58.98	87.62	90.90	55.32	88.86	58.62	57.95	90.19	57.71	56.72	91.91	54.53	55.56	58.62	57.17	59.58	55.76
848	372	16	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	88.93	60.69	89.11	62.31	61.29	92.39	89.49	90.54	90.01	90.95	91.90	59.43	90.55	91.72	60.34	89.58	60.63	90.87	91.06	91.33	57.27	60.05	59.84	58.69	58.48
851	372	16	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	89.05	61.65	61.72	61.05	89.20	92.43	89.28	90.98	90.91	90.22	92.05	58.88	91.96	90.61	58.45	92.97	58.84	89.71	91.67	92.46	58.04	58.95	90.25	58.07	57.61
860	372	16	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	88.90	62.53	87.99	90.66	90.47	90.93	87.86	59.29	89.62	60.25	58.37	58.48	92.17	59.91	60.84	59.80	59.86	60.41	91.02	92.19	58.96	59.97	54.65	59.87	57.09
863	372	16	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	87.91	89.66	87.92	60.66	89.35	89.70	89.82	60.72	91.34	91.72	56.78	93.72	89.99	59.55	60.05	91.21	60.87	91.44	60.25	88.70	58.13	57.64	83.08	60.50	58.35
912	372	16	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	88.66	62.22	87.68	89.15	89.60	92.71	86.81	89.41	90.41	82.57	56.24	59.21	91.43	92.19	59.78	90.91	90.92	91.72	92.69	58.52	55.65	56.70	57.43	59.25	57.08
915	372	16	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	89.90	60.69	89.19	55.25	88.08	57.39	87.85	89.52	86.32	90.68	89.88	57.43	89.93	58.78	58.55	85.80	91.42	90.02	91.30	58.07	58.94	57.85	57.21	58.07	58.33
924	372	16	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	90.20	88.68	63.47	88.68	89.20	90.90	85.79	59.15	89.41	90.71	91.40	56.33	89.61	90.71	60.27	91.19	59.66	89.66	90.66	59.17	56.23	56.34	57.28	56.38	59.20
927	372	16	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	88.46	87.09	89.91	60.96	89.22	91.97	85.83	87.38	90.35	91.21	58.52	60.25	87.27	89.10	60.31	88.72	59.70	88.67	90.63	91.17	58.57	57.66	88.05	57.70	59.86
976	372	16	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	64.30	64.42	62.49	63.59	62.48	89.37	64.66	63.25	58.27	64.37	57.83	59.84	60.15	59.84	62.40	58.79	58.69	64.58	57.33	43.72	65.89	66.09	57.47	63.64	66.20
979	372	16	0.1	16	0.00010	48	0.000000e+00	1.000000e-08	62.60	61.93	87.15	60.65	63.55	91.07	56.53	59.89	56.96	62.40	92.01	56.22	89.86	60.64	61.12	91.38	61.89	59.81	91.15	55.99	60.81	61.56	62.31	60.39	58.96
988	372	16	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	56.80	63.55	63.33	63.42	62.59	89.44	88.48	62.25	61.09	62.98	90.24	59.35	88.30	61.79	64.19	57.31	89.13	91.87	89.89	58.51	61.79	65.03	63.99	60.93	62.23
991	372	16	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	86.34	61.55	61.41	63.31	59.70	91.78	90.43	61.42	62.89	62.13	57.53	91.55	88.65	57.96	61.89	60.37	61.47	57.22	90.66	55.01	60.40	61.95	59.56	60.63	60.19
1040	372	16	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	88.89	61.87	61.98	61.42	62.74	59.13	89.46	60.32	63.03	92.13	91.40	58.59	89.81	91.97	60.77	92.84	61.27	92.13	59.04	56.06	60.51	58.34	56.97	58.34	60.46
1043	372	16	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	88.74	61.54	56.02	60.76	90.23	91.55	90.83	89.93	88.03	90.48	92.94	92.21	92.17	90.13	60.70	90.16	91.23	61.51	91.68	58.99	60.37	59.38	91.31	59.32	59.64
1052	372	16	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	89.24	56.29	89.18	89.89	90.07	92.20	88.39	86.86	90.51	91.39	91.67	91.40	85.96	91.69	91.75	91.73	92.33	91.14	91.92	93.30	57.34	60.20	90.91	59.53	58.61
1055	372	16	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	89.79	88.87	61.26	87.91	60.72	91.94	89.82	43.72	60.86	91.47	59.19	59.41	90.29	59.40	61.53	91.86	61.60	92.22	90.67	59.25	57.45	59.03	53.78	59.37	58.21
1104	372	16	0.1	64	0.00010	48	0.000000e+00	0.000000e+00	88.94	89.41	88.09	60.53	89.18	58.75	90.67	58.64	90.22	90.21	56.03	55.53	91.15	91.95	58.86	86.42	58.86	91.04	58.60	56.45	57.40	59.21	58.41	58.58	59.14
1107	372	16	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	87.39	89.69	88.72	61.07	90.19	92.22	86.85	59.25	90.69	59.55	90.25	58.05	90.35	91.92	88.77	91.27	60.30	90.42	90.62	90.63	57.06	57.47	57.67	59.67	57.11
1116	372	16	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	88.17	62.07	89.02	60.28	90.69	90.38	89.91	60.50	58.85	59.21	89.03	57.12	91.80	58.10	59.08	88.90	59.93	56.06	90.41	58.76	57.73	56.81	57.31	56.93	56.57
1119	372	16	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	87.39	61.63	89.42	90.72	90.93	90.62	88.92	60.44	89.36	89.04	89.74	91.42	91.67	90.17	59.04	87.13	92.17	88.38	91.42	57.77	56.45	57.74	57.84	56.86	87.50
1168	372	16	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	43.72	86.61	82.98	81.68	68.79	86.52	61.33	87.47	88.01	84.65	84.05	86.96	85.02	64.27	86.83	69.19	89.09	86.49	61.20	86.18	87.60	87.23	79.69	86.08	77.73
1171	372	16	0.1	16	0.00001	24	0.000000e+00	1.000000e-08	84.51	82.65	84.15	81.96	84.15	88.72	84.51	84.17	83.85	88.69	86.22	87.65	86.56	83.61	85.33	86.84	86.23	85.80	90.47	86.81	63.05	62.24	85.51	86.44	88.10
1180	372	16	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	83.22	64.41	84.00	87.39	63.16	88.61	84.64	84.08	85.61	84.07	87.91	90.75	81.83	60.11	62.26	86.41	87.25	85.16	88.99	88.70	88.51	87.81	87.75	82.30	81.99
1183	372	16	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	68.97	82.92	82.30	85.51	83.96	84.85	82.52	86.20	83.53	85.83	86.25	86.55	86.76	84.38	87.37	86.09	87.45	85.74	84.19	88.01	86.41	85.81	87.17	83.19	89.32
1232	372	16	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	84.34	88.50	85.00	85.11	85.54	89.43	85.04	90.36	84.49	87.40	87.07	88.56	85.46	88.96	88.85	87.65	87.07	87.46	88.05	90.28	89.89	90.24	87.06	88.88	88.73
1235	372	16	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	89.05	88.73	84.44	87.67	88.65	88.47	85.52	86.18	85.17	88.63	88.15	86.98	83.02	87.81	88.07	87.52	87.67	88.27	88.94	88.83	88.47	89.77	88.96	89.16	89.64
1244	372	16	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	87.09	56.26	90.39	87.56	86.18	87.13	85.10	86.80	84.84	87.05	89.42	89.47	86.52	87.40	90.50	89.57	90.68	87.68	87.89	92.34	87.49	87.16	89.89	89.12	88.16
1247	372	16	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	84.58	84.59	86.32	85.49	86.47	87.79	85.51	86.81	87.52	87.79	90.17	90.89	85.04	87.85	85.19	86.80	88.55	86.10	91.11	91.00	86.91	90.07	89.11	89.46	91.74
1296	372	16	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	83.64	87.25	86.17	86.01	84.04	84.35	84.32	83.76	86.34	85.06	90.10	87.14	86.00	87.57	88.43	84.72	88.75	86.13	91.93	88.54	90.19	87.55	89.01	88.92	86.52
1299	372	16	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	83.56	88.14	86.09	87.56	87.27	90.65	87.25	84.45	85.00	87.33	88.41	88.13	85.74	87.66	86.01	86.00	86.45	85.91	88.19	87.20	87.66	88.40	87.12	92.66	87.26
1308	372	16	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	84.51	84.40	86.44	87.67	84.63	89.60	85.43	86.81	87.52	88.85	92.35	85.87	88.10	85.37	85.00	89.16	88.67	85.77	89.07	87.68	87.83	87.89	86.97	94.91	89.60
1311	372	16	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	84.86	88.30	86.21	84.12	84.60	86.03	85.42	88.34	86.13	86.73	91.75	88.47	88.15	86.42	87.87	87.67	86.31	87.77	91.86	87.95	89.04	89.03	87.72	88.34	90.22
1360	372	16	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	65.58	58.27	63.72	62.16	66.03	61.30	57.31	62.30	63.58	56.73	58.93	59.46	56.50	60.51	62.23	89.82	64.50	61.51	59.40	61.48	61.34	65.02	61.17	63.06	60.89
1363	372	16	0.1	16	0.00001	36	0.000000e+00	1.000000e-08	86.68	63.88	65.93	63.06	64.68	61.81	56.96	64.69	64.13	66.59	92.05	59.93	88.88	57.86	61.29	89.97	62.27	62.90	59.98	60.25	62.06	63.92	61.99	62.30	62.32
1372	372	16	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	57.86	65.78	64.76	67.98	65.03	62.65	87.90	65.15	88.33	64.51	89.80	62.55	66.01	64.79	64.42	64.11	64.40	90.16	88.78	62.84	63.24	63.25	63.21	62.42	63.53
1375	372	16	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	56.77	61.88	63.38	63.50	61.57	58.50	86.71	60.34	61.03	62.58	57.80	59.27	89.48	89.17	60.00	61.11	61.71	63.91	58.36	55.43	61.94	65.24	59.60	59.80	58.82
1424	372	16	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	89.43	59.99	60.86	90.31	60.51	91.97	89.47	60.49	89.18	60.16	92.47	57.77	92.64	91.10	60.33	93.08	83.96	90.79	91.56	90.95	91.54	58.70	57.63	58.23	59.40
1427	372	16	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	88.67	62.08	87.03	61.46	61.26	90.62	88.58	60.67	90.54	61.60	91.46	92.21	90.92	91.47	63.02	91.14	60.73	91.16	92.97	58.62	59.34	59.07	56.94	60.05	59.26
1436	372	16	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	88.43	61.33	88.44	88.80	88.55	89.92	87.44	61.95	62.37	91.44	91.14	90.45	87.65	91.46	60.53	90.42	61.43	91.18	58.35	91.29	58.42	58.54	61.91	61.55	59.67
1439	372	16	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	88.47	89.82	88.79	59.01	91.19	91.66	91.21	60.82	90.74	58.80	90.26	92.18	90.96	59.26	60.79	91.55	90.82	92.10	89.06	92.93	60.76	57.98	59.48	57.82	58.15
1488	372	16	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	89.14	61.27	88.17	89.93	61.05	87.32	90.16	60.64	92.20	88.96	89.61	81.35	90.16	60.86	58.95	85.71	58.00	92.71	90.14	57.22	57.08	56.61	89.86	57.30	55.88
1491	372	16	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	84.69	61.17	59.80	60.26	88.17	91.97	89.41	91.15	91.95	89.42	88.79	89.80	90.45	90.92	59.30	91.43	58.10	90.67	91.19	90.92	58.06	55.24	58.52	58.95	59.23
1500	372	16	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	90.70	89.65	88.40	60.42	88.96	91.64	89.95	59.91	87.61	90.44	90.13	88.29	88.04	59.64	60.36	92.22	58.65	90.68	56.51	59.00	56.15	59.04	57.28	56.17	59.28
1503	372	16	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	89.43	89.17	88.15	89.41	89.32	90.86	86.01	58.79	90.54	59.20	56.87	59.43	88.27	61.32	59.48	92.18	59.51	90.44	93.47	92.47	89.77	58.18	57.56	56.73	58.39
1552	372	16	0.1	16	0.00001	48	0.000000e+00	0.000000e+00	86.08	61.67	63.42	62.95	61.64	57.52	88.58	60.37	63.58	57.38	55.33	55.67	89.75	57.85	88.19	50.04	59.44	61.82	58.14	58.44	58.88	63.04	62.08	53.90	59.58
1555	372	16	0.1	16	0.00001	48	0.000000e+00	1.000000e-08	87.84	64.96	87.53	63.11	62.04	60.35	86.82	62.65	89.56	62.66	91.72	91.02	90.55	59.14	60.37	57.25	88.64	89.20	92.44	58.25	60.02	60.21	53.54	60.03	61.05
1564	372	16	0.1	16	0.00001	48	1.000000e-08	0.000000e+00	56.48	57.80	86.59	86.95	60.66	91.31	88.38	54.71	57.75	89.61	90.40	90.80	89.08	57.76	55.06	91.24	58.57	56.12	92.22	93.17	52.31	57.73	57.41	57.25	56.02
1567	372	16	0.1	16	0.00001	48	1.000000e-08	1.000000e-08	87.95	60.61	62.06	88.21	61.51	88.26	87.73	60.61	62.48	57.09	90.67	55.07	91.51	60.59	62.04	58.14	60.20	60.46	58.37	91.90	60.47	58.40	59.38	58.50	59.53

Test Data mAP Results Preview:

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_abs_values	mAP_test_abs_values_2	mAP_test_abs_values_3	mAP_test_abs_values_4	mAP_test_abs_values_5	mAP_test_abs_values_6	mAP_test_abs_values_7	mAP_test_abs_values_8	mAP_test_abs_values_9	mAP_test_abs_values_10	mAP_test_abs_values_11	mAP_test_abs_values_12	mAP_test_abs_values_13	mAP_test_abs_values_14	mAP_test_abs_values_15	mAP_test_abs_values_16	mAP_test_abs_values_17	mAP_test_abs_values_18	mAP_test_abs_values_19	mAP_test_abs_values_20	mAP_test_abs_values_21	mAP_test_abs_values_22	mAP_test_abs_values_23	mAP_test_abs_values_25	mAP_test_abs_values_26
16	372	16	0.1	16	0.00100	24	0.000000e+00	0.000000e+00	89.22	53.30	91.55	91.08	90.85	90.51	91.25	88.72	92.76	53.84	90.44	80.52	90.89	89.17	90.55	89.99	91.31	88.56	83.71	84.41	60.99	85.81	62.34	86.87	92.61
19	372	16	0.1	16	0.00100	24	0.000000e+00	1.000000e-08	94.06	50.60	90.28	80.17	84.53	91.42	93.10	64.11	90.48	91.57	94.69	92.42	92.58	91.61	66.41	92.10	60.19	92.18	92.48	89.05	90.44	57.27	61.36	91.58	89.09
28	372	16	0.1	16	0.00100	24	1.000000e-08	0.000000e+00	88.71	90.80	92.75	61.10	93.34	92.91	54.02	93.55	89.37	82.92	92.02	89.36	91.72	90.64	81.89	87.90	88.70	89.09	90.92	87.63	93.23	92.14	53.92	92.51	91.63
31	372	16	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	93.24	85.06	90.90	52.01	86.35	92.83	93.28	91.87	92.31	92.53	89.52	91.56	91.45	93.15	92.27	92.25	48.12	91.66	90.31	88.51	88.86	91.17	92.11	91.60	93.42
80	372	16	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	90.14	95.59	89.47	92.91	93.64	91.47	93.72	94.63	95.72	95.10	93.32	86.65	93.49	48.12	58.31	78.61	94.49	91.95	92.81	91.60	60.28	92.39	93.12	92.61	91.61
83	372	16	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	94.19	90.90	90.84	91.66	95.26	91.57	94.89	93.40	94.35	93.48	92.71	92.47	92.06	94.79	93.24	95.37	90.54	91.11	92.75	92.35	58.52	92.53	91.17	92.05	93.54
92	372	16	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	94.28	90.49	93.53	91.49	93.49	92.08	94.27	93.27	92.27	91.95	93.36	92.92	91.38	94.82	87.56	93.06	93.74	93.11	92.68	94.12	92.64	95.06	91.86	52.49	59.55
95	372	16	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	95.06	89.72	91.66	93.80	92.61	91.53	93.76	93.01	92.28	92.95	91.69	93.94	94.12	91.35	94.42	93.33	96.19	92.82	92.79	92.19	93.98	93.16	94.17	92.58	68.98
144	372	16	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	94.36	92.56	94.32	91.82	95.50	90.05	92.67	93.12	94.56	93.18	92.63	92.37	94.68	94.64	88.37	92.70	48.12	93.22	89.68	91.30	91.74	92.36	92.46	56.53	92.61
147	372	16	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	94.33	96.37	91.19	93.37	95.37	92.15	95.78	93.31	93.31	93.08	93.84	92.27	91.78	87.67	48.12	94.38	48.12	90.83	92.11	92.25	93.78	93.33	93.26	91.47	91.64
156	372	16	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	90.71	94.11	90.14	92.09	94.61	92.87	93.48	91.67	94.93	94.47	92.71	92.29	95.04	92.02	94.50	94.52	93.27	94.86	92.32	92.34	93.98	94.02	94.16	92.60	90.30
159	372	16	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	92.55	88.17	93.56	87.90	93.08	93.26	92.77	93.30	94.07	93.37	91.79	93.07	91.98	95.89	94.92	93.92	92.10	92.87	93.10	91.17	93.96	92.72	92.46	90.35	93.19
208	372	16	0.1	16	0.00100	36	0.000000e+00	0.000000e+00	58.69	59.22	60.49	61.89	62.06	60.45	49.39	60.53	60.71	60.21	59.70	60.83	59.11	59.92	61.42	49.53	59.66	59.76	58.34	59.97	62.93	61.82	62.82	61.35	61.92
211	372	16	0.1	16	0.00100	36	0.000000e+00	1.000000e-08	57.92	60.92	50.36	61.07	48.12	60.32	59.07	60.84	48.91	59.60	60.76	60.69	48.12	59.24	59.90	57.97	60.24	59.32	58.52	60.03	60.69	48.12	61.71	62.88	61.42
220	372	16	0.1	16	0.00100	36	1.000000e-08	0.000000e+00	50.38	61.18	59.50	61.21	61.77	49.64	58.89	62.02	58.50	61.30	61.85	62.34	58.52	61.04	60.28	59.94	48.24	59.30	61.09	50.39	60.74	60.17	51.59	62.06	62.08
223	372	16	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	57.54	49.78	59.67	50.36	61.29	58.27	59.22	61.43	59.17	59.22	58.94	59.01	57.98	58.30	60.36	57.79	60.11	57.24	57.32	60.47	61.31	60.98	61.99	62.66	62.45
272	372	16	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	59.13	61.71	61.17	61.65	62.19	59.84	49.92	50.31	93.02	61.47	60.00	60.01	60.50	59.16	61.99	59.22	60.74	56.91	58.82	61.51	61.90	61.47	59.94	62.71	62.64
275	372	16	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	60.15	61.26	61.69	61.53	58.55	60.12	92.14	59.13	61.76	90.45	50.54	61.93	59.76	60.81	61.86	59.53	49.50	59.81	59.95	59.61	61.78	61.22	62.20	62.65	62.26
284	372	16	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	59.56	50.77	59.58	61.70	61.99	59.85	59.30	62.70	59.29	61.99	59.71	59.47	59.91	58.97	61.07	59.24	48.63	58.33	59.95	59.65	61.43	62.59	60.43	62.79	61.67
287	372	16	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	58.39	59.87	59.27	62.34	58.04	59.76	59.26	60.81	59.23	59.02	60.27	61.99	59.34	58.97	59.44	88.21	61.26	59.34	62.44	59.67	62.48	61.76	61.64	62.04	62.83
336	372	16	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	60.37	61.99	60.27	61.93	59.31	59.63	59.95	62.88	59.57	61.76	60.22	60.94	61.31	61.03	61.50	51.78	61.61	59.56	61.13	59.16	61.45	62.35	62.26	62.32	62.44
339	372	16	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	59.69	59.80	59.77	62.57	60.33	59.88	50.94	62.16	60.37	59.71	61.20	62.06	91.19	62.41	61.99	59.90	59.43	59.79	62.57	62.64	62.60	62.55	59.72	62.66	62.64
348	372	16	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	59.86	63.04	59.77	61.99	59.31	59.76	60.29	62.26	91.13	59.79	62.64	60.11	50.28	59.85	59.95	59.86	62.09	59.23	51.59	59.99	62.55	62.34	59.05	62.43	62.30
351	372	16	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	59.85	62.64	60.19	62.06	59.95	60.77	59.53	59.72	59.83	60.09	59.90	61.20	59.62	89.99	48.12	51.38	59.66	59.65	62.64	60.67	60.26	63.04	62.79	62.77	61.68
400	372	16	0.1	16	0.00100	48	0.000000e+00	0.000000e+00	58.23	61.43	50.98	61.27	60.37	50.98	60.84	60.58	60.96	59.97	59.64	61.03	48.19	60.63	48.12	59.02	61.75	61.13	61.31	60.53	62.71	62.28	61.75	61.67	61.20
403	372	16	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	50.10	61.25	50.23	50.64	60.89	49.58	50.64	60.23	50.52	57.10	58.12	60.42	57.96	60.56	60.63	59.07	61.31	59.07	49.90	49.74	61.49	61.34	60.03	48.12	61.05
412	372	16	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	48.12	58.35	60.46	61.51	60.25	50.25	57.90	61.76	48.44	61.18	59.94	61.09	59.37	61.40	60.58	60.51	60.87	61.61	58.66	61.08	62.45	63.77	62.60	62.36	62.32
415	372	16	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	50.25	50.45	59.81	60.23	59.84	58.92	57.83	60.34	56.68	59.47	51.52	58.82	57.73	59.40	59.78	50.37	57.42	57.43	56.98	51.21	60.41	60.89	52.17	61.27	61.66
464	372	16	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	59.76	62.32	50.58	48.12	62.09	49.74	58.89	61.53	59.13	61.42	62.64	59.62	59.38	59.24	60.89	59.72	61.50	61.76	61.65	61.29	62.55	61.16	62.76	62.88	61.65
467	372	16	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	59.49	62.32	59.59	61.22	61.99	60.44	59.45	61.42	62.00	59.13	60.27	61.99	59.52	60.95	62.00	59.87	60.67	58.39	60.01	84.45	61.85	61.88	53.93	61.82	61.95
476	372	16	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	59.57	61.99	58.86	59.80	59.61	49.46	57.71	62.06	50.24	59.36	61.70	61.75	59.30	58.53	61.18	58.88	61.08	59.76	60.48	62.16	61.28	62.09	60.38	62.72	61.88
479	372	16	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	59.23	60.81	59.32	62.32	59.43	59.79	59.22	61.54	50.51	58.64	59.95	61.59	59.54	60.61	59.34	59.42	59.30	89.36	59.52	58.53	60.79	63.09	60.02	63.04	63.27
528	372	16	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	59.56	59.80	62.32	59.50	60.12	60.01	60.08	60.50	59.69	59.86	59.71	62.84	60.73	61.53	61.70	60.53	61.24	59.58	62.97	62.06	62.66	63.04	62.57	62.03	62.53
531	372	16	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	59.95	60.17	59.47	59.36	59.57	61.30	60.00	61.52	60.83	60.03	61.07	62.41	60.29	60.08	61.48	60.33	61.01	60.27	59.80	88.26	60.26	63.02	62.39	62.36	61.87
540	372	16	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	60.84	61.99	60.12	61.93	59.77	62.90	59.95	63.26	51.66	61.71	61.13	62.26	60.94	61.20	61.99	60.84	61.23	60.03	51.00	62.32	50.26	62.82	62.87	63.16	62.84
543	372	16	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	59.91	62.39	62.16	60.38	60.26	60.87	60.34	62.41	62.41	62.09	62.32	59.95	60.09	62.75	60.77	59.33	61.76	59.71	60.32	51.11	62.32	62.66	62.75	62.52	62.97
592	372	16	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	90.47	83.21	90.72	92.21	92.15	91.22	90.39	90.30	91.95	55.97	93.62	90.16	90.76	57.93	54.12	90.37	84.59	90.25	83.34	90.40	88.62	58.61	57.12	82.44	57.02
595	372	16	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	91.26	91.18	94.19	91.70	93.66	88.79	80.84	85.43	93.07	95.10	89.31	92.46	93.02	92.00	76.02	89.59	90.87	91.85	91.48	92.35	91.87	73.49	90.57	90.22	92.64
604	372	16	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	92.75	95.16	92.22	92.15	90.71	85.03	86.10	91.03	93.39	90.33	89.77	92.31	92.13	91.62	59.76	93.17	93.83	53.12	92.76	92.00	92.24	89.44	91.63	92.27	91.23
607	372	16	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	91.42	89.71	93.65	87.12	90.62	91.85	94.29	90.54	89.90	91.12	92.60	88.77	89.97	90.32	59.15	89.51	59.15	91.24	85.68	87.29	91.10	90.42	90.18	91.86	59.31
656	372	16	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	92.69	64.70	95.15	83.55	93.84	89.19	94.58	90.22	92.09	91.73	92.08	88.52	90.55	92.48	88.57	92.15	85.81	92.30	91.34	91.90	94.27	88.95	91.27	91.97	92.87
659	372	16	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	92.95	91.59	94.03	92.10	93.67	91.16	93.56	92.03	92.80	94.10	91.70	93.22	90.51	93.23	85.61	93.76	91.67	94.15	91.63	93.87	94.04	92.72	91.05	59.53	92.98
668	372	16	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	90.08	89.78	95.16	91.27	94.40	92.24	93.73	59.22	89.69	94.23	93.56	90.63	95.43	92.18	93.14	93.92	91.36	92.30	92.24	94.37	93.13	92.95	91.65	60.70	92.84
671	372	16	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	94.85	93.37	94.58	93.60	88.49	92.41	95.60	91.14	93.18	93.32	93.02	91.45	94.45	94.91	93.96	90.87	90.43	93.69	92.31	89.46	90.15	58.98	93.92	92.34	91.91
720	372	16	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	92.67	94.38	93.26	91.17	94.49	92.91	91.39	89.88	93.59	93.35	94.72	93.54	94.80	88.14	93.49	91.57	91.68	93.77	91.13	92.37	61.03	93.21	94.43	93.94	89.94
723	372	16	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	95.16	92.56	88.22	90.20	91.91	92.76	94.47	94.19	92.58	92.09	92.14	91.85	92.77	94.70	51.18	94.56	94.38	92.64	92.37	93.06	61.67	92.36	91.43	94.84	92.74
732	372	16	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	92.88	91.41	89.96	48.12	93.76	93.15	93.96	91.19	93.65	92.58	93.68	90.68	92.24	90.56	95.45	87.57	91.94	92.89	90.93	91.75	91.13	91.46	93.71	92.72	91.75
735	372	16	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	95.00	93.69	95.54	91.07	89.48	92.31	93.35	92.64	92.31	93.84	89.18	94.39	90.38	92.86	91.41	93.57	90.27	89.99	92.21	92.87	92.24	92.54	61.11	92.86	70.78
784	372	16	0.1	16	0.00010	36	0.000000e+00	0.000000e+00	49.86	60.88	60.49	60.11	61.56	61.62	57.51	60.71	49.69	61.18	60.90	50.70	57.60	60.62	60.15	62.32	60.48	61.29	59.91	60.67	62.05	61.05	61.18	61.68	62.21
787	372	16	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	57.42	61.60	60.77	60.66	60.86	59.81	58.00	59.51	58.49	60.68	51.40	61.95	90.11	59.51	61.20	59.22	60.87	59.63	61.31	58.29	62.28	62.75	62.08	61.96	61.91
796	372	16	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	58.55	61.58	49.53	61.77	60.99	58.04	58.93	61.07	58.67	60.20	59.43	61.65	58.92	60.86	60.22	60.98	60.37	57.33	60.47	61.28	61.32	61.77	61.63	61.62	61.46
799	372	16	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	58.28	48.94	59.91	59.32	59.95	57.94	56.61	60.04	58.72	56.54	58.92	49.92	57.55	59.53	59.10	56.94	59.98	48.57	57.70	50.00	59.72	60.00	63.10	61.09	60.31
848	372	16	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	59.79	61.28	58.64	61.99	61.93	59.98	58.63	59.17	58.51	59.12	59.33	62.32	59.43	59.56	61.65	59.57	60.29	58.74	59.80	59.43	62.48	61.56	62.34	62.99	61.94
851	372	16	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	59.99	61.99	61.65	61.93	59.41	59.71	59.19	59.84	58.19	58.54	60.06	62.70	59.14	59.61	61.48	59.13	62.56	59.64	60.15	58.56	60.63	61.83	59.88	62.78	62.10
860	372	16	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	58.97	61.98	59.41	59.81	59.25	59.80	52.02	63.04	59.57	61.83	61.99	62.06	59.62	62.55	61.10	60.72	60.99	61.70	60.39	59.91	61.91	62.01	52.12	62.44	62.66
863	372	16	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	59.39	59.15	59.57	62.39	60.27	60.02	59.38	62.32	59.42	59.56	62.97	59.32	59.63	60.72	61.93	59.58	61.70	59.52	61.93	50.96	62.34	62.57	87.88	62.15	62.13
912	372	16	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	59.91	62.06	59.85	60.22	59.62	59.19	92.87	60.03	60.08	90.63	50.51	61.71	59.62	59.21	62.06	60.37	59.90	59.58	60.27	61.99	62.25	62.70	62.39	61.78	62.75
915	372	16	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	59.88	62.32	59.57	50.86	59.43	62.64	93.02	59.96	51.47	59.61	60.67	62.07	60.09	61.99	61.67	91.97	59.81	59.57	60.01	61.99	62.09	62.92	62.75	62.66	61.62
924	372	16	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	59.45	59.70	61.74	59.72	59.21	60.33	61.20	62.41	59.81	59.50	51.47	62.72	60.34	58.92	61.87	59.85	61.61	59.98	60.22	62.32	61.55	62.45	62.79	62.50	61.81
927	372	16	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	59.42	52.56	59.93	62.39	59.46	59.33	60.53	50.74	60.65	59.57	62.97	61.93	51.36	60.72	61.76	50.40	61.66	60.86	60.83	59.53	62.36	62.57	60.44	63.09	62.44
976	372	16	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	59.29	61.33	61.34	61.19	60.65	57.11	61.44	61.06	49.13	60.88	59.18	61.60	48.63	60.15	60.71	49.14	63.79	60.59	50.39	48.12	60.94	61.17	51.32	61.94	61.50
979	372	16	0.1	16	0.00010	48	0.000000e+00	1.000000e-08	60.48	60.78	57.58	61.44	61.76	57.84	50.00	60.39	49.14	61.32	59.15	49.82	58.51	60.91	60.65	57.98	60.52	59.35	58.87	49.82	62.17	61.81	62.02	61.69	62.06
988	372	16	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	50.20	61.29	61.34	61.78	60.30	58.67	58.29	61.98	60.87	61.34	60.30	61.12	59.23	59.49	60.26	50.37	59.31	58.67	58.28	61.05	61.87	61.39	61.51	62.96	62.38
991	372	16	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	57.86	60.77	60.52	61.33	59.70	47.99	58.47	60.85	60.41	58.75	60.00	56.86	57.71	58.70	60.66	60.13	59.26	50.62	56.55	50.01	61.76	61.80	61.63	61.96	60.99
1040	372	16	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	59.29	61.65	61.65	61.65	61.36	61.93	59.11	61.36	61.88	59.57	60.38	61.91	59.00	58.98	61.93	58.56	61.64	58.77	62.32	61.38	61.74	62.41	62.10	62.32	61.43
1043	372	16	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	59.62	61.93	50.73	61.93	59.34	59.86	59.41	59.86	52.41	59.37	59.68	59.80	59.67	58.37	61.11	60.27	58.97	61.45	59.96	61.99	61.22	61.92	59.43	62.46	61.81
1052	372	16	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	59.47	49.98	58.86	60.19	60.23	59.78	57.95	51.19	59.11	59.57	60.38	60.49	52.74	60.08	59.35	58.63	58.51	59.69	59.90	59.87	62.55	60.78	59.69	61.83	62.45
1055	372	16	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	59.78	59.92	61.68	60.38	61.59	59.83	59.00	48.12	61.93	59.83	62.64	62.32	59.92	60.66	61.74	59.28	61.60	58.92	60.50	61.87	62.67	62.21	52.63	62.24	62.26
1104	372	16	0.1	64	0.00010	48	0.000000e+00	0.000000e+00	59.04	60.27	59.46	61.70	59.75	62.64	59.57	62.34	59.52	59.55	62.97	63.04	59.77	59.62	60.89	50.93	61.77	59.12	62.71	62.32	62.90	61.76	62.26	61.93	62.12
1107	372	16	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	60.43	59.65	59.41	61.99	59.81	59.45	60.40	62.90	59.46	61.76	60.73	62.32	60.60	59.86	52.81	59.54	61.64	59.61	60.77	60.77	62.48	62.39	62.60	62.32	62.97
1116	372	16	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	59.94	62.26	60.58	61.93	59.86	60.61	59.87	62.05	62.64	62.20	61.43	62.64	59.62	62.64	61.01	60.38	62.19	49.50	60.42	61.93	62.27	62.43	62.27	62.73	62.97
1119	372	16	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	60.32	61.01	59.62	59.48	59.80	60.77	59.48	62.26	60.77	60.89	49.99	60.00	59.57	60.03	61.45	91.04	59.71	60.81	60.43	61.94	62.91	62.03	62.71	62.68	51.89
1168	372	16	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	48.12	90.56	92.71	91.27	93.16	88.44	92.28	90.24	92.52	92.41	84.56	86.40	87.33	89.62	50.95	89.78	93.52	90.78	91.15	90.95	88.54	91.25	91.96	92.29	89.59
1171	372	16	0.1	16	0.00001	24	0.000000e+00	1.000000e-08	93.16	89.98	93.70	88.50	95.29	57.42	92.12	59.26	94.02	92.40	92.53	90.37	91.80	75.70	75.60	93.80	57.73	91.60	76.96	89.83	89.69	90.63	93.72	87.32	90.03
1180	372	16	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	91.77	91.94	92.46	92.74	89.65	92.41	94.63	59.41	93.56	92.98	94.22	52.88	90.72	93.91	89.48	92.57	92.32	58.94	90.40	88.90	92.71	92.74	59.18	90.67	55.75
1183	372	16	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	90.49	91.92	92.19	90.23	93.71	88.94	89.80	92.80	92.53	92.38	88.92	88.49	94.35	92.67	90.45	90.89	58.20	88.97	91.24	87.27	92.24	89.98	92.43	53.81	59.90
1232	372	16	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	92.35	90.96	94.04	89.83	88.15	91.19	93.21	91.17	93.41	89.31	93.20	87.84	94.32	92.77	70.79	94.44	91.10	90.53	90.15	91.43	83.08	92.39	93.26	91.48	90.78
1235	372	16	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	90.95	86.68	93.58	67.88	93.04	92.25	93.49	91.38	90.20	94.45	92.70	89.92	91.61	92.15	89.66	92.60	95.88	94.86	94.65	91.40	93.40	93.46	93.06	59.93	91.23
1244	372	16	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	91.80	92.49	93.06	92.59	94.92	93.93	93.16	93.44	89.70	93.43	91.94	91.10	90.16	91.81	90.04	93.73	92.41	91.51	93.67	91.80	92.76	86.25	92.01	91.46	91.58
1247	372	16	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	94.20	93.47	93.99	92.52	94.83	95.10	91.75	93.41	95.11	94.35	92.90	91.74	89.98	89.75	93.46	93.76	95.16	92.90	93.48	93.37	91.10	92.29	93.46	91.12	88.63
1296	372	16	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	94.60	93.94	93.65	92.35	92.43	93.48	93.65	91.89	77.31	90.65	92.09	93.45	93.81	94.04	93.52	84.88	94.88	94.63	93.08	91.34	94.16	85.82	93.25	91.45	85.43
1299	372	16	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	94.23	92.84	94.31	95.50	91.99	93.21	95.25	95.92	94.37	91.72	93.25	92.23	93.72	94.74	94.36	96.20	91.42	92.78	92.41	94.19	94.51	93.18	92.28	60.84	93.55
1308	372	16	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	94.05	91.38	94.72	93.19	90.28	93.68	93.75	92.66	92.64	94.20	93.23	94.81	94.08	92.23	91.67	92.96	92.14	94.84	94.51	92.27	92.53	91.61	89.13	61.90	87.80
1311	372	16	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	93.36	92.41	92.03	90.34	90.39	93.05	94.18	93.77	92.96	93.39	91.64	93.58	92.92	95.46	93.51	91.97	90.96	92.04	92.89	93.30	91.80	95.07	88.88	93.09	92.51
1360	372	16	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	61.20	48.90	60.54	60.40	60.93	61.34	49.88	60.83	60.71	50.70	60.59	59.25	49.76	59.62	61.57	59.39	61.79	59.63	59.49	61.96	62.40	61.23	62.75	61.42	62.41
1363	372	16	0.1	16	0.00001	36	0.000000e+00	1.000000e-08	59.97	60.73	61.04	61.79	61.10	61.37	50.31	61.29	60.75	61.11	59.03	62.32	59.16	49.63	61.01	58.19	60.57	58.76	61.64	61.99	62.38	61.91	61.84	62.13	62.17
1372	372	16	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	49.11	61.59	61.38	60.99	61.73	61.47	59.39	60.01	58.65	59.75	60.49	61.65	61.10	61.00	60.76	60.65	61.59	58.57	48.75	62.06	61.81	62.12	61.83	62.03	62.23
1375	372	16	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	50.12	59.99	60.61	61.01	60.13	60.13	56.90	59.66	60.09	59.84	60.22	61.37	58.15	57.37	60.06	61.70	60.86	60.41	60.56	50.83	62.05	61.43	60.52	62.54	61.52
1424	372	16	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	58.84	61.05	61.47	59.25	61.92	59.43	58.95	61.83	58.57	61.42	59.62	61.93	59.42	59.83	61.41	59.28	90.43	58.21	58.16	58.63	59.16	62.87	62.67	62.50	61.72
1427	372	16	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	59.49	61.87	59.93	61.72	61.94	60.61	59.14	62.14	58.97	61.99	59.25	59.89	59.67	59.52	60.57	59.87	62.28	59.01	59.48	61.90	62.32	62.55	50.39	61.94	62.03
1436	372	16	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	59.18	62.08	59.37	59.20	59.79	59.92	59.14	61.71	62.06	59.41	59.90	59.58	59.99	58.74	61.76	59.68	61.56	59.33	62.64	59.88	62.45	62.55	61.60	62.46	61.76
1439	372	16	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	58.99	59.72	59.33	61.59	59.20	59.78	59.23	62.15	58.61	61.98	60.84	59.90	59.14	61.95	62.06	59.43	59.84	58.74	51.80	59.95	61.56	62.22	61.65	62.39	62.03
1488	372	16	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	60.04	61.75	59.80	59.81	61.99	50.27	59.95	61.77	59.55	59.34	60.77	88.17	59.69	62.41	62.64	90.69	60.96	58.88	60.43	62.26	62.46	63.04	59.57	62.34	62.61
1491	372	16	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	93.25	62.32	62.64	61.79	59.80	59.18	59.69	60.13	59.70	59.57	61.20	61.07	59.33	59.91	60.90	59.71	61.32	59.57	60.22	60.32	62.55	63.11	62.72	61.64	62.26
1500	372	16	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	59.14	60.32	59.82	62.64	59.19	60.37	59.53	61.76	51.03	59.61	60.67	61.26	60.80	60.72	62.64	59.23	60.95	59.56	62.97	62.32	62.72	61.39	62.17	62.80	61.33
1503	372	16	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	59.52	59.79	59.14	59.95	59.80	60.87	52.25	62.97	59.37	62.03	62.97	62.39	60.97	61.21	62.32	59.61	61.70	59.71	59.80	59.62	60.25	62.75	62.48	62.72	62.03
1552	372	16	0.1	16	0.00001	48	0.000000e+00	0.000000e+00	56.79	61.64	60.38	61.10	60.45	59.50	57.68	60.34	61.32	48.94	50.75	50.24	58.48	59.44	58.31	56.28	60.48	58.35	60.30	60.28	61.13	61.90	61.11	64.42	62.86
1555	372	16	0.1	16	0.00001	48	0.000000e+00	1.000000e-08	58.39	61.11	59.19	61.59	62.09	61.25	58.63	61.99	58.21	60.83	60.33	59.81	58.16	60.52	60.03	50.14	57.63	58.19	59.09	60.97	63.03	61.31	51.90	62.47	62.28
1564	372	16	0.1	16	0.00001	48	1.000000e-08	0.000000e+00	50.46	57.43	56.65	55.72	60.53	58.33	57.19	52.14	49.03	57.76	58.32	59.64	57.98	59.86	48.93	57.13	58.73	58.01	58.37	58.31	50.77	59.92	61.14	60.04	60.77
1567	372	16	0.1	16	0.00001	48	1.000000e-08	1.000000e-08	57.48	60.85	60.90	59.45	61.99	56.69	58.34	60.70	60.73	51.49	58.10	50.55	58.39	60.96	60.44	49.33	58.31	58.67	60.63	58.73	61.13	61.66	61.14	62.64	60.67

Model performance on validation data (T-test)

Statistical Significance:

Using the global mean maximum row as the reference, we perform the right-tailed t-test to identify significant hyperparameters.

pvalues = []
pvalues_real = []

for _,index in enumerate(dt_valid_sub.index):
  _, p_value = stats.ttest_ind(max_row[:-1], dt_valid_sub.loc[index][:-1], alternative='greater')
  pvalues_real.append(p_value)
  pvalues.append((p_value >= 0.05)*1)

  

dt_valid_sub['pvalues'] = pvalues_real
dt_valid_sub['sig'] = pvalues


dt_valid_sub1 = dt_valid_sub.query('sig == 1')


max_indexx = dt_valid_sub1.iloc[:, :-3].mean().idxmax()

#print(sum(pvalues),max_indexx)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_abs_values	mAP_valid_abs_values_2	mAP_valid_abs_values_3	mAP_valid_abs_values_4	mAP_valid_abs_values_5	mAP_valid_abs_values_6	mAP_valid_abs_values_7	mAP_valid_abs_values_8	mAP_valid_abs_values_9	mAP_valid_abs_values_10	mAP_valid_abs_values_11	mAP_valid_abs_values_12	mAP_valid_abs_values_13	mAP_valid_abs_values_14	mAP_valid_abs_values_15	mAP_valid_abs_values_16	mAP_valid_abs_values_17	mAP_valid_abs_values_18	mAP_valid_abs_values_19	mAP_valid_abs_values_20	mAP_valid_abs_values_21	mAP_valid_abs_values_22	mAP_valid_abs_values_23	mAP_valid_abs_values_25	mAP_valid_abs_values_26
1232	372	16	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	84.34	88.50	85.00	85.11	85.54	89.43	85.04	90.36	84.49	87.40	87.07	88.56	85.46	88.96	88.85	87.65	87.07	87.46	88.05	90.28	89.89	90.24	87.06	88.88	88.73
1235	372	16	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	89.05	88.73	84.44	87.67	88.65	88.47	85.52	86.18	85.17	88.63	88.15	86.98	83.02	87.81	88.07	87.52	87.67	88.27	88.94	88.83	88.47	89.77	88.96	89.16	89.64
1244	372	16	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	87.09	56.26	90.39	87.56	86.18	87.13	85.10	86.80	84.84	87.05	89.42	89.47	86.52	87.40	90.50	89.57	90.68	87.68	87.89	92.34	87.49	87.16	89.89	89.12	88.16
1247	372	16	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	84.58	84.59	86.32	85.49	86.47	87.79	85.51	86.81	87.52	87.79	90.17	90.89	85.04	87.85	85.19	86.80	88.55	86.10	91.11	91.00	86.91	90.07	89.11	89.46	91.74
659	372	16	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	84.52	86.55	85.38	84.63	82.42	85.52	83.98	86.72	85.40	90.50	86.34	89.77	88.63	91.02	86.47	87.33	86.67	86.96	89.29	91.82	88.34	89.81	88.81	91.15	88.36
668	372	16	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	85.59	86.09	86.06	86.18	84.20	87.39	83.80	89.21	86.84	86.47	90.30	86.26	86.55	86.31	88.75	88.86	89.61	86.83	90.76	88.24	89.45	89.32	89.08	91.03	88.43
671	372	16	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	83.26	86.23	86.92	86.67	87.02	88.96	83.94	87.96	85.68	87.44	86.99	88.83	86.02	88.41	87.44	88.89	86.44	87.11	90.42	87.46	87.76	89.84	90.64	88.32	88.15
1052	372	16	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	89.24	56.29	89.18	89.89	90.07	92.20	88.39	86.86	90.51	91.39	91.67	91.40	85.96	91.69	91.75	91.73	92.33	91.14	91.92	93.30	57.34	60.20	90.91	59.53	58.61
80	372	16	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	84.83	88.42	84.03	84.61	87.24	84.92	84.39	87.86	87.97	85.69	90.03	86.42	87.25	43.72	90.43	86.88	86.93	88.62	86.36	87.33	89.49	88.90	87.10	89.67	86.19
83	372	16	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	86.14	85.10	87.21	86.01	86.26	86.64	87.30	86.68	88.58	85.51	89.28	85.47	89.02	88.46	89.49	88.29	88.45	89.81	90.12	88.13	89.61	88.90	89.81	89.34	89.20
92	372	16	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	85.66	55.61	86.21	87.07	82.88	86.35	83.86	88.87	90.69	84.39	87.88	92.49	87.78	85.49	90.85	87.91	89.03	90.32	88.94	88.65	87.86	88.46	89.81	89.91	89.77
95	372	16	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	84.67	85.95	84.21	89.47	85.77	90.28	87.24	88.50	86.75	87.52	84.52	53.51	85.93	86.60	89.88	87.80	86.05	86.87	88.55	89.62	87.66	88.76	89.07	87.97	91.66
1299	372	16	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	83.56	88.14	86.09	87.56	87.27	90.65	87.25	84.45	85.00	87.33	88.41	88.13	85.74	87.66	86.01	86.00	86.45	85.91	88.19	87.20	87.66	88.40	87.12	92.66	87.26
1308	372	16	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	84.51	84.40	86.44	87.67	84.63	89.60	85.43	86.81	87.52	88.85	92.35	85.87	88.10	85.37	85.00	89.16	88.67	85.77	89.07	87.68	87.83	87.89	86.97	94.91	89.60
1311	372	16	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	84.86	88.30	86.21	84.12	84.60	86.03	85.42	88.34	86.13	86.73	91.75	88.47	88.15	86.42	87.87	87.67	86.31	87.77	91.86	87.95	89.04	89.03	87.72	88.34	90.22
720	372	16	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	84.13	86.57	83.83	87.60	84.34	86.64	86.54	85.90	88.58	84.97	87.53	88.43	87.43	86.89	87.65	86.53	90.35	88.53	90.74	91.37	94.45	88.25	88.48	88.75	86.88
723	372	16	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	87.23	89.30	86.63	87.65	86.57	91.51	84.03	89.13	89.00	87.01	90.99	87.75	87.29	88.16	89.22	86.77	87.58	88.77	88.82	93.96	91.15	89.34	87.55	56.13	90.02
732	372	16	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	82.88	85.26	85.21	43.72	84.03	88.06	84.79	88.69	85.21	85.28	90.98	87.30	88.21	88.46	85.99	88.31	86.74	86.63	85.75	87.24	88.77	89.08	88.29	89.33	89.03
735	372	16	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	84.30	85.92	84.63	84.90	82.97	88.29	84.36	87.53	86.64	87.78	90.68	87.85	89.74	86.51	88.48	85.88	90.90	85.08	89.24	89.57	90.12	89.68	88.58	90.50	91.32
144	372	16	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	84.32	84.79	86.49	86.62	85.24	88.61	85.96	88.21	87.19	87.61	88.45	88.65	86.93	87.94	88.30	55.46	43.72	88.27	87.74	89.04	88.92	88.51	89.34	91.75	89.32
147	372	16	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	86.38	85.67	83.84	89.78	86.49	84.83	86.89	87.94	85.84	88.76	87.55	91.69	87.17	87.06	43.72	86.51	43.72	89.07	89.58	90.44	88.87	88.90	88.62	90.41	88.78
156	372	16	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	84.03	85.49	84.62	84.11	89.00	89.08	85.88	84.64	85.14	85.31	91.59	89.31	86.01	86.88	87.95	88.05	87.15	87.36	88.27	87.60	89.02	89.00	89.47	90.10	90.79
159	372	16	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	86.63	84.66	84.54	83.76	86.59	85.53	86.50	86.99	87.35	88.42	87.92	90.59	88.28	86.22	87.39	88.12	87.10	88.33	86.72	93.02	89.38	90.40	87.25	90.93	90.07

Size of the All data:  (100, 28)
Size of the Sig data:  (23, 28)

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params1 = dt_valid.loc[dt_valid_sub1.index][['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
optimal_params = optimal_params1.reset_index()
optimal_params.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_1 = pd.merge(optimal_params, dt_test, on=['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg'])
test_data_1_sub = test_data_1[list(test_data_1.columns[test_data_1.columns.str.startswith('mAP_test')])]

Equivalent test data

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_abs_values	mAP_test_abs_values_2	mAP_test_abs_values_3	mAP_test_abs_values_4	mAP_test_abs_values_5	mAP_test_abs_values_6	mAP_test_abs_values_7	mAP_test_abs_values_8	mAP_test_abs_values_9	mAP_test_abs_values_10	mAP_test_abs_values_11	mAP_test_abs_values_12	mAP_test_abs_values_13	mAP_test_abs_values_14	mAP_test_abs_values_15	mAP_test_abs_values_16	mAP_test_abs_values_17	mAP_test_abs_values_18	mAP_test_abs_values_19	mAP_test_abs_values_20	mAP_test_abs_values_21	mAP_test_abs_values_22	mAP_test_abs_values_23	mAP_test_abs_values_25	mAP_test_abs_values_26
8	372	16	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	92.35	90.96	94.04	89.83	88.15	91.19	93.21	91.17	93.41	89.31	93.20	87.84	94.32	92.77	70.79	94.44	91.10	90.53	90.15	91.43	83.08	92.39	93.26	91.48	90.78
1	372	16	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	90.95	86.68	93.58	67.88	93.04	92.25	93.49	91.38	90.20	94.45	92.70	89.92	91.61	92.15	89.66	92.60	95.88	94.86	94.65	91.40	93.40	93.46	93.06	59.93	91.23
15	372	16	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	91.80	92.49	93.06	92.59	94.92	93.93	93.16	93.44	89.70	93.43	91.94	91.10	90.16	91.81	90.04	93.73	92.41	91.51	93.67	91.80	92.76	86.25	92.01	91.46	91.58
2	372	16	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	94.20	93.47	93.99	92.52	94.83	95.10	91.75	93.41	95.11	94.35	92.90	91.74	89.98	89.75	93.46	93.76	95.16	92.90	93.48	93.37	91.10	92.29	93.46	91.12	88.63
11	372	16	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	92.95	91.59	94.03	92.10	93.67	91.16	93.56	92.03	92.80	94.10	91.70	93.22	90.51	93.23	85.61	93.76	91.67	94.15	91.63	93.87	94.04	92.72	91.05	59.53	92.98
4	372	16	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	90.08	89.78	95.16	91.27	94.40	92.24	93.73	59.22	89.69	94.23	93.56	90.63	95.43	92.18	93.14	93.92	91.36	92.30	92.24	94.37	93.13	92.95	91.65	60.70	92.84
10	372	16	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	94.85	93.37	94.58	93.60	88.49	92.41	95.60	91.14	93.18	93.32	93.02	91.45	94.45	94.91	93.96	90.87	90.43	93.69	92.31	89.46	90.15	58.98	93.92	92.34	91.91
22	372	16	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	59.47	49.98	58.86	60.19	60.23	59.78	57.95	51.19	59.11	59.57	60.38	60.49	52.74	60.08	59.35	58.63	58.51	59.69	59.90	59.87	62.55	60.78	59.69	61.83	62.45
18	372	16	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	90.14	95.59	89.47	92.91	93.64	91.47	93.72	94.63	95.72	95.10	93.32	86.65	93.49	48.12	58.31	78.61	94.49	91.95	92.81	91.60	60.28	92.39	93.12	92.61	91.61
0	372	16	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	94.19	90.90	90.84	91.66	95.26	91.57	94.89	93.40	94.35	93.48	92.71	92.47	92.06	94.79	93.24	95.37	90.54	91.11	92.75	92.35	58.52	92.53	91.17	92.05	93.54
16	372	16	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	94.28	90.49	93.53	91.49	93.49	92.08	94.27	93.27	92.27	91.95	93.36	92.92	91.38	94.82	87.56	93.06	93.74	93.11	92.68	94.12	92.64	95.06	91.86	52.49	59.55
17	372	16	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	95.06	89.72	91.66	93.80	92.61	91.53	93.76	93.01	92.28	92.95	91.69	93.94	94.12	91.35	94.42	93.33	96.19	92.82	92.79	92.19	93.98	93.16	94.17	92.58	68.98
14	372	16	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	94.23	92.84	94.31	95.50	91.99	93.21	95.25	95.92	94.37	91.72	93.25	92.23	93.72	94.74	94.36	96.20	91.42	92.78	92.41	94.19	94.51	93.18	92.28	60.84	93.55
7	372	16	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	94.05	91.38	94.72	93.19	90.28	93.68	93.75	92.66	92.64	94.20	93.23	94.81	94.08	92.23	91.67	92.96	92.14	94.84	94.51	92.27	92.53	91.61	89.13	61.90	87.80
9	372	16	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	93.36	92.41	92.03	90.34	90.39	93.05	94.18	93.77	92.96	93.39	91.64	93.58	92.92	95.46	93.51	91.97	90.96	92.04	92.89	93.30	91.80	95.07	88.88	93.09	92.51
6	372	16	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	92.67	94.38	93.26	91.17	94.49	92.91	91.39	89.88	93.59	93.35	94.72	93.54	94.80	88.14	93.49	91.57	91.68	93.77	91.13	92.37	61.03	93.21	94.43	93.94	89.94
13	372	16	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	95.16	92.56	88.22	90.20	91.91	92.76	94.47	94.19	92.58	92.09	92.14	91.85	92.77	94.70	51.18	94.56	94.38	92.64	92.37	93.06	61.67	92.36	91.43	94.84	92.74
19	372	16	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	92.88	91.41	89.96	48.12	93.76	93.15	93.96	91.19	93.65	92.58	93.68	90.68	92.24	90.56	95.45	87.57	91.94	92.89	90.93	91.75	91.13	91.46	93.71	92.72	91.75
5	372	16	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	95.00	93.69	95.54	91.07	89.48	92.31	93.35	92.64	92.31	93.84	89.18	94.39	90.38	92.86	91.41	93.57	90.27	89.99	92.21	92.87	92.24	92.54	61.11	92.86	70.78
20	372	16	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	94.36	92.56	94.32	91.82	95.50	90.05	92.67	93.12	94.56	93.18	92.63	92.37	94.68	94.64	88.37	92.70	48.12	93.22	89.68	91.30	91.74	92.36	92.46	56.53	92.61
21	372	16	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	94.33	96.37	91.19	93.37	95.37	92.15	95.78	93.31	93.31	93.08	93.84	92.27	91.78	87.67	48.12	94.38	48.12	90.83	92.11	92.25	93.78	93.33	93.26	91.47	91.64
12	372	16	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	90.71	94.11	90.14	92.09	94.61	92.87	93.48	91.67	94.93	94.47	92.71	92.29	95.04	92.02	94.50	94.52	93.27	94.86	92.32	92.34	93.98	94.02	94.16	92.60	90.30
3	372	16	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	92.55	88.17	93.56	87.90	93.08	93.26	92.77	93.30	94.07	93.37	91.79	93.07	91.98	95.89	94.92	93.92	92.10	92.87	93.10	91.17	93.96	92.72	92.46	90.35	93.19

Size of the test data:  (23, 33)

Difference

diff = dt[['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
for i, cols in enumerate(zip(dt.columns,test_data_1.columns)):   
   diff[f'col_diff_{i}'] = (np.array(dt[cols[0]]) - np.array(test_data_1[cols[1]]))*-1

# Function to count negatives in a row
def count_negatives(row):
    return sum(1 for value in row if isinstance(value, (int, float)) and value < 0)

diff['negative_count'] = diff.apply(count_negatives, axis=1)
diff.sort_values(by =["negative_count"],inplace = True, ascending=False)

html_table = diff.to_html(index=True)

# Wrap in a scrollable div
scrollable_table = f"""
<div style="height: 400px; width: 100%; overflow-x: auto; overflow-y: auto;">
    {html_table}
</div>
"""
# Display the scrollable table
display(HTML(scrollable_table))

# 
# show(diff)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	col_diff_0	col_diff_1	col_diff_2	col_diff_3	col_diff_4	col_diff_5	col_diff_6	col_diff_7	col_diff_8	col_diff_9	col_diff_10	col_diff_11	col_diff_12	col_diff_13	col_diff_14	col_diff_15	col_diff_16	col_diff_17	col_diff_18	col_diff_19	col_diff_20	col_diff_21	col_diff_22	col_diff_23	col_diff_24	col_diff_25	col_diff_26	col_diff_27	col_diff_28	col_diff_29	col_diff_30	col_diff_31	col_diff_32	negative_count
1052	372	16	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.77	-6.31	-30.32	-29.70	-29.84	-32.42	-30.44	-35.67	-31.40	-31.82	-31.29	-30.91	-33.22	-31.61	-32.40	-33.10	-33.82	-31.45	-32.02	-33.43	5.21	0.58	-31.22	2.30	3.84	21
735	372	16	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	10.70	7.77	10.91	6.17	6.51	4.02	8.99	5.11	5.67	6.06	-1.50	6.54	0.64	6.35	2.93	7.69	-0.63	4.91	2.97	3.30	2.12	2.86	-27.47	2.36	-20.54	4
1232	372	16	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.01	2.46	9.04	4.72	2.61	1.76	8.17	0.81	8.92	1.91	6.13	-0.72	8.86	3.81	-18.06	6.79	4.03	3.07	2.10	1.15	-6.81	2.15	6.20	2.60	2.05	3
92	372	16	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.62	34.88	7.32	4.42	10.61	5.73	10.41	4.40	1.58	7.56	5.48	0.43	3.60	9.33	-3.29	5.15	4.71	2.79	3.74	5.47	4.78	6.60	2.05	-37.42	-30.22	3
1244	372	16	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	4.71	36.23	2.67	5.03	8.74	6.80	8.06	6.64	4.86	6.38	2.52	1.63	3.64	4.41	-0.46	4.16	1.73	3.83	5.78	-0.54	5.27	-0.91	2.12	2.34	3.42	3
723	372	16	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.93	3.26	1.59	2.55	5.34	1.25	10.44	5.06	3.58	5.08	1.15	4.10	5.48	6.54	-38.04	7.79	6.80	3.87	3.55	-0.90	-29.48	3.02	3.88	38.71	2.72	3
1235	372	16	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	1.90	-2.05	9.14	-19.79	4.39	3.78	7.97	5.20	5.03	5.82	4.55	2.94	8.59	4.34	1.59	5.08	8.21	6.59	5.71	2.57	4.93	3.69	4.10	-29.23	1.59	3
80	372	16	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.31	7.17	5.44	8.30	6.40	6.55	9.33	6.77	7.75	9.41	3.29	0.23	6.24	4.40	-32.12	-8.27	7.56	3.33	6.45	4.27	-29.21	3.49	6.02	2.94	5.42	3
1308	372	16	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.54	6.98	8.28	5.52	5.65	4.08	8.32	5.85	5.12	5.35	0.88	8.94	5.98	6.86	6.67	3.80	3.47	9.07	5.44	4.59	4.70	3.72	2.16	-33.01	-1.80	2
159	372	16	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.92	3.51	9.02	4.14	6.49	7.73	6.27	6.31	6.72	4.95	3.87	2.48	3.70	9.67	7.53	5.80	5.00	4.54	6.38	-1.85	4.58	2.32	5.21	-0.58	3.12	2
668	372	16	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	4.49	3.69	9.10	5.09	10.20	4.85	9.93	-29.99	2.85	7.76	3.26	4.37	8.88	5.87	4.39	5.06	1.75	5.47	1.48	6.13	3.68	3.63	2.57	-30.33	4.41	2
659	372	16	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.43	5.04	8.65	7.47	11.25	5.64	9.58	5.31	7.40	3.60	5.36	3.45	1.88	2.21	-0.86	6.43	5.00	7.19	2.34	2.05	5.70	2.91	2.24	-31.62	4.62	2
83	372	16	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.05	5.80	3.63	5.65	9.00	4.93	7.59	6.72	5.77	7.97	3.43	7.00	3.04	6.33	3.75	7.08	2.09	1.30	2.63	4.22	-31.09	3.63	1.36	2.71	4.34	1
671	372	16	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	11.59	7.14	7.66	6.93	1.47	3.45	11.66	3.18	7.50	5.88	6.03	2.62	8.43	6.50	6.52	1.98	3.99	6.58	1.89	2.00	2.39	-30.86	3.28	4.02	3.76	1
1299	372	16	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	10.67	4.70	8.22	7.94	4.72	2.56	8.00	11.47	9.37	4.39	4.84	4.10	7.98	7.08	8.35	10.20	4.97	6.87	4.22	6.99	6.85	4.78	5.16	-31.82	6.29	1
1311	372	16	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.50	4.11	5.82	6.22	5.79	7.02	8.76	5.43	6.83	6.66	-0.11	5.11	4.77	9.04	5.64	4.30	4.65	4.27	1.03	5.35	2.76	6.04	1.16	4.75	2.29	1
720	372	16	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.54	7.81	9.43	3.57	10.15	6.27	4.85	3.98	5.01	8.38	7.19	5.11	7.37	1.25	5.84	5.04	1.33	5.24	0.39	1.00	-33.42	4.96	5.95	5.19	3.06	1
732	372	16	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	10.00	6.15	4.75	4.40	9.73	5.09	9.17	2.50	8.44	7.30	2.70	3.38	4.03	2.10	9.46	-0.74	5.20	6.26	5.18	4.51	2.36	2.38	5.42	3.39	2.72	1
1247	372	16	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.62	8.88	7.67	7.03	8.36	7.31	6.24	6.60	7.59	6.56	2.73	0.85	4.94	1.90	8.27	6.96	6.61	6.80	2.37	2.37	4.19	2.22	4.35	1.66	-3.11	1
144	372	16	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	10.04	7.77	7.83	5.20	10.26	1.44	6.71	4.91	7.37	5.57	4.18	3.72	7.75	6.70	0.07	37.24	4.40	4.95	1.94	2.26	2.82	3.85	3.12	-35.22	3.29	1
156	372	16	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.68	8.62	5.52	7.98	5.61	3.79	7.60	7.03	9.79	9.16	1.12	2.98	9.03	5.14	6.55	6.47	6.12	7.50	4.05	4.74	4.96	5.02	4.69	2.50	-0.49	1
95	372	16	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	10.39	3.77	7.45	4.33	6.84	1.25	6.52	4.51	5.53	5.43	7.17	40.43	8.19	4.75	4.54	5.53	10.14	5.95	4.24	2.57	6.32	4.40	5.10	4.61	-22.68	1
147	372	16	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.95	10.70	7.35	3.59	8.88	7.32	8.89	5.37	7.47	4.32	6.29	0.58	4.61	0.61	4.40	7.87	4.40	1.76	2.53	1.81	4.91	4.43	4.64	1.06	2.86	0

Model performance on valid data

Average & Std Deviation of the Significant rows:

	mean	std
mAP_valid_abs_values_20	89.655217	2.135358
mAP_valid_abs_values_11	89.131304	2.025645
mAP_valid_abs_values_19	89.057826	1.635459
mAP_valid_abs_values_23	88.680000	1.156248
mAP_valid_abs_values_6	87.996087	2.054004
mAP_valid_abs_values_26	87.910000	6.544522
mAP_valid_abs_values_22	87.822174	6.071457
mAP_valid_abs_values_18	87.767826	1.487316
mAP_valid_abs_values_21	87.629565	6.781426
mAP_valid_abs_values_8	87.453913	1.429039
mAP_valid_abs_values_10	87.296957	1.706470
mAP_valid_abs_values_25	87.276087	9.434357
mAP_valid_abs_values_12	87.134348	7.566842
mAP_valid_abs_values_13	86.966522	1.493454
mAP_valid_abs_values_9	86.871304	1.755372
mAP_valid_abs_values_16	86.421304	6.872286
mAP_valid_abs_values_15	86.315217	9.454895
mAP_valid_abs_values_5	85.844783	1.951053
mAP_valid_abs_values_3	85.820870	1.634909
mAP_valid_abs_values_14	85.708261	9.278572
mAP_valid_abs_values_7	85.526957	1.313200
mAP_valid_abs_values	85.295652	1.676590
mAP_valid_abs_values_4	84.689130	9.109113
mAP_valid_abs_values_17	84.268261	12.905762
mAP_valid_abs_values_2	82.470435	10.563067

Model performance on Test data

We then apply the hyperparameters to the test set and average the results.

test_data_1['average_map'] = test_data_1_sub.apply(np.mean, axis=1)

test_data_1.sort_values(by=['average_map'], ascending= False,inplace=True)

	mean	std
mAP_test_abs_values_7	92.180000	7.536976
mAP_test_abs_values_10	91.804783	7.129978
mAP_test_abs_values	91.722609	7.205630
mAP_test_abs_values_9	91.599565	7.257929
mAP_test_abs_values_5	91.460435	7.149847
mAP_test_abs_values_3	91.306522	7.345817
mAP_test_abs_values_18	91.276087	7.014224
mAP_test_abs_values_11	91.273478	6.822132
mAP_test_abs_values_16	91.130435	7.902112
mAP_test_abs_values_13	91.071304	8.520884
mAP_test_abs_values_6	91.048261	6.899788
mAP_test_abs_values_19	90.987826	6.878546
mAP_test_abs_values_20	90.986957	6.881164
mAP_test_abs_values_12	90.584783	6.836328
mAP_test_abs_values_2	90.213043	9.049217
mAP_test_abs_values_22	89.774783	9.579237
mAP_test_abs_values_23	89.640435	9.342427
average_map	89.586748	6.818341
mAP_test_abs_values_8	89.519130	10.974914
mAP_test_abs_values_14	89.342174	11.479508
mAP_test_abs_values_4	87.591739	11.918222
mAP_test_abs_values_17	87.212174	14.325746
mAP_test_abs_values_26	87.082174	10.440165
mAP_test_abs_values_21	85.391304	13.456053
mAP_test_abs_values_15	84.631304	15.268350
mAP_test_abs_values_25	80.837391	16.268674

Model performance on validation data (Mann-Whitney U Test)

Statistical Significance:

Using the global mean maximum row as the reference, we perform the right-tailed t-test to identify significant hyperparameters.

pvalues_mw = []
pvalues_real_mw = []

for _,index in enumerate(dt_valid_sub.index):

  _, p_value_mw = stats.mannwhitneyu(max_row[:-1], dt_valid_sub.loc[index][:-1], alternative='greater')#, method = 'asymptotic')
  pvalues_real_mw.append(p_value_mw)
  pvalues_mw.append((p_value_mw >= 0.05)*1)
  

dt_valid_sub_mw['pvalues_mw'] = pvalues_real_mw
dt_valid_sub_mw['sig_mw'] = pvalues_mw

dt_valid_sub_mw1 = dt_valid_sub_mw.query('sig_mw == 1')


max_indexx = dt_valid_sub_mw1.iloc[:, :-3].mean().idxmax()

#print(sum(pvalues),max_indexx)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_abs_values	mAP_valid_abs_values_2	mAP_valid_abs_values_3	mAP_valid_abs_values_4	mAP_valid_abs_values_5	mAP_valid_abs_values_6	mAP_valid_abs_values_7	mAP_valid_abs_values_8	mAP_valid_abs_values_9	mAP_valid_abs_values_10	mAP_valid_abs_values_11	mAP_valid_abs_values_12	mAP_valid_abs_values_13	mAP_valid_abs_values_14	mAP_valid_abs_values_15	mAP_valid_abs_values_16	mAP_valid_abs_values_17	mAP_valid_abs_values_18	mAP_valid_abs_values_19	mAP_valid_abs_values_20	mAP_valid_abs_values_21	mAP_valid_abs_values_22	mAP_valid_abs_values_23	mAP_valid_abs_values_25	mAP_valid_abs_values_26
607	372	16	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	67.50	69.28	88.19	88.72	88.74	85.57	83.91	90.79	90.45	90.32	88.88	65.01	87.98	90.31	84.46	83.94	87.54	86.43	82.93	86.99	90.74	88.20	88.15	89.14	88.23
1232	372	16	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	84.34	88.50	85.00	85.11	85.54	89.43	85.04	90.36	84.49	87.40	87.07	88.56	85.46	88.96	88.85	87.65	87.07	87.46	88.05	90.28	89.89	90.24	87.06	88.88	88.73
1235	372	16	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	89.05	88.73	84.44	87.67	88.65	88.47	85.52	86.18	85.17	88.63	88.15	86.98	83.02	87.81	88.07	87.52	87.67	88.27	88.94	88.83	88.47	89.77	88.96	89.16	89.64
1244	372	16	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	87.09	56.26	90.39	87.56	86.18	87.13	85.10	86.80	84.84	87.05	89.42	89.47	86.52	87.40	90.50	89.57	90.68	87.68	87.89	92.34	87.49	87.16	89.89	89.12	88.16
1247	372	16	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	84.58	84.59	86.32	85.49	86.47	87.79	85.51	86.81	87.52	87.79	90.17	90.89	85.04	87.85	85.19	86.80	88.55	86.10	91.11	91.00	86.91	90.07	89.11	89.46	91.74
1424	372	16	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	89.43	59.99	60.86	90.31	60.51	91.97	89.47	60.49	89.18	60.16	92.47	57.77	92.64	91.10	60.33	93.08	83.96	90.79	91.56	90.95	91.54	58.70	57.63	58.23	59.40
1427	372	16	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	88.67	62.08	87.03	61.46	61.26	90.62	88.58	60.67	90.54	61.60	91.46	92.21	90.92	91.47	63.02	91.14	60.73	91.16	92.97	58.62	59.34	59.07	56.94	60.05	59.26
1436	372	16	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	88.43	61.33	88.44	88.80	88.55	89.92	87.44	61.95	62.37	91.44	91.14	90.45	87.65	91.46	60.53	90.42	61.43	91.18	58.35	91.29	58.42	58.54	61.91	61.55	59.67
1439	372	16	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	88.47	89.82	88.79	59.01	91.19	91.66	91.21	60.82	90.74	58.80	90.26	92.18	90.96	59.26	60.79	91.55	90.82	92.10	89.06	92.93	60.76	57.98	59.48	57.82	58.15
659	372	16	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	84.52	86.55	85.38	84.63	82.42	85.52	83.98	86.72	85.40	90.50	86.34	89.77	88.63	91.02	86.47	87.33	86.67	86.96	89.29	91.82	88.34	89.81	88.81	91.15	88.36
668	372	16	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	85.59	86.09	86.06	86.18	84.20	87.39	83.80	89.21	86.84	86.47	90.30	86.26	86.55	86.31	88.75	88.86	89.61	86.83	90.76	88.24	89.45	89.32	89.08	91.03	88.43
671	372	16	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	83.26	86.23	86.92	86.67	87.02	88.96	83.94	87.96	85.68	87.44	86.99	88.83	86.02	88.41	87.44	88.89	86.44	87.11	90.42	87.46	87.76	89.84	90.64	88.32	88.15
848	372	16	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	88.93	60.69	89.11	62.31	61.29	92.39	89.49	90.54	90.01	90.95	91.90	59.43	90.55	91.72	60.34	89.58	60.63	90.87	91.06	91.33	57.27	60.05	59.84	58.69	58.48
851	372	16	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	89.05	61.65	61.72	61.05	89.20	92.43	89.28	90.98	90.91	90.22	92.05	58.88	91.96	90.61	58.45	92.97	58.84	89.71	91.67	92.46	58.04	58.95	90.25	58.07	57.61
863	372	16	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	87.91	89.66	87.92	60.66	89.35	89.70	89.82	60.72	91.34	91.72	56.78	93.72	89.99	59.55	60.05	91.21	60.87	91.44	60.25	88.70	58.13	57.64	83.08	60.50	58.35
1043	372	16	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	88.74	61.54	56.02	60.76	90.23	91.55	90.83	89.93	88.03	90.48	92.94	92.21	92.17	90.13	60.70	90.16	91.23	61.51	91.68	58.99	60.37	59.38	91.31	59.32	59.64
1052	372	16	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	89.24	56.29	89.18	89.89	90.07	92.20	88.39	86.86	90.51	91.39	91.67	91.40	85.96	91.69	91.75	91.73	92.33	91.14	91.92	93.30	57.34	60.20	90.91	59.53	58.61
83	372	16	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	86.14	85.10	87.21	86.01	86.26	86.64	87.30	86.68	88.58	85.51	89.28	85.47	89.02	88.46	89.49	88.29	88.45	89.81	90.12	88.13	89.61	88.90	89.81	89.34	89.20
92	372	16	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	85.66	55.61	86.21	87.07	82.88	86.35	83.86	88.87	90.69	84.39	87.88	92.49	87.78	85.49	90.85	87.91	89.03	90.32	88.94	88.65	87.86	88.46	89.81	89.91	89.77
95	372	16	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	84.67	85.95	84.21	89.47	85.77	90.28	87.24	88.50	86.75	87.52	84.52	53.51	85.93	86.60	89.88	87.80	86.05	86.87	88.55	89.62	87.66	88.76	89.07	87.97	91.66
284	372	16	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	88.12	56.48	90.11	62.18	62.70	90.43	89.46	59.56	91.34	61.07	91.97	92.70	89.76	90.04	61.86	91.19	55.31	91.03	91.43	92.35	59.89	58.29	89.92	59.07	58.83
287	372	16	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	88.39	90.65	88.85	61.19	88.63	92.22	89.61	59.68	91.54	91.49	91.05	58.79	90.50	91.45	92.65	82.95	59.15	90.62	58.93	91.19	56.89	58.14	59.40	60.00	57.57
479	372	16	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	87.95	61.87	90.22	61.29	88.97	92.04	91.37	59.78	56.52	90.71	92.44	57.58	90.51	58.92	89.45	91.93	91.73	83.90	92.47	90.16	88.72	55.90	89.65	57.97	56.25
1308	372	16	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	84.51	84.40	86.44	87.67	84.63	89.60	85.43	86.81	87.52	88.85	92.35	85.87	88.10	85.37	85.00	89.16	88.67	85.77	89.07	87.68	87.83	87.89	86.97	94.91	89.60
1311	372	16	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	84.86	88.30	86.21	84.12	84.60	86.03	85.42	88.34	86.13	86.73	91.75	88.47	88.15	86.42	87.87	87.67	86.31	87.77	91.86	87.95	89.04	89.03	87.72	88.34	90.22
1488	372	16	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	89.14	61.27	88.17	89.93	61.05	87.32	90.16	60.64	92.20	88.96	89.61	81.35	90.16	60.86	58.95	85.71	58.00	92.71	90.14	57.22	57.08	56.61	89.86	57.30	55.88
1491	372	16	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	84.69	61.17	59.80	60.26	88.17	91.97	89.41	91.15	91.95	89.42	88.79	89.80	90.45	90.92	59.30	91.43	58.10	90.67	91.19	90.92	58.06	55.24	58.52	58.95	59.23
1500	372	16	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	90.70	89.65	88.40	60.42	88.96	91.64	89.95	59.91	87.61	90.44	90.13	88.29	88.04	59.64	60.36	92.22	58.65	90.68	56.51	59.00	56.15	59.04	57.28	56.17	59.28
1503	372	16	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	89.43	89.17	88.15	89.41	89.32	90.86	86.01	58.79	90.54	59.20	56.87	59.43	88.27	61.32	59.48	92.18	59.51	90.44	93.47	92.47	89.77	58.18	57.56	56.73	58.39
720	372	16	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	84.13	86.57	83.83	87.60	84.34	86.64	86.54	85.90	88.58	84.97	87.53	88.43	87.43	86.89	87.65	86.53	90.35	88.53	90.74	91.37	94.45	88.25	88.48	88.75	86.88
723	372	16	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	87.23	89.30	86.63	87.65	86.57	91.51	84.03	89.13	89.00	87.01	90.99	87.75	87.29	88.16	89.22	86.77	87.58	88.77	88.82	93.96	91.15	89.34	87.55	56.13	90.02
735	372	16	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	84.30	85.92	84.63	84.90	82.97	88.29	84.36	87.53	86.64	87.78	90.68	87.85	89.74	86.51	88.48	85.88	90.90	85.08	89.24	89.57	90.12	89.68	88.58	90.50	91.32
912	372	16	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	88.66	62.22	87.68	89.15	89.60	92.71	86.81	89.41	90.41	82.57	56.24	59.21	91.43	92.19	59.78	90.91	90.92	91.72	92.69	58.52	55.65	56.70	57.43	59.25	57.08
924	372	16	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	90.20	88.68	63.47	88.68	89.20	90.90	85.79	59.15	89.41	90.71	91.40	56.33	89.61	90.71	60.27	91.19	59.66	89.66	90.66	59.17	56.23	56.34	57.28	56.38	59.20
927	372	16	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	88.46	87.09	89.91	60.96	89.22	91.97	85.83	87.38	90.35	91.21	58.52	60.25	87.27	89.10	60.31	88.72	59.70	88.67	90.63	91.17	58.57	57.66	88.05	57.70	59.86
1107	372	16	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	87.39	89.69	88.72	61.07	90.19	92.22	86.85	59.25	90.69	59.55	90.25	58.05	90.35	91.92	88.77	91.27	60.30	90.42	90.62	90.63	57.06	57.47	57.67	59.67	57.11
1119	372	16	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	87.39	61.63	89.42	90.72	90.93	90.62	88.92	60.44	89.36	89.04	89.74	91.42	91.67	90.17	59.04	87.13	92.17	88.38	91.42	57.77	56.45	57.74	57.84	56.86	87.50
144	372	16	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	84.32	84.79	86.49	86.62	85.24	88.61	85.96	88.21	87.19	87.61	88.45	88.65	86.93	87.94	88.30	55.46	43.72	88.27	87.74	89.04	88.92	88.51	89.34	91.75	89.32
147	372	16	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	86.38	85.67	83.84	89.78	86.49	84.83	86.89	87.94	85.84	88.76	87.55	91.69	87.17	87.06	43.72	86.51	43.72	89.07	89.58	90.44	88.87	88.90	88.62	90.41	88.78
156	372	16	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	84.03	85.49	84.62	84.11	89.00	89.08	85.88	84.64	85.14	85.31	91.59	89.31	86.01	86.88	87.95	88.05	87.15	87.36	88.27	87.60	89.02	89.00	89.47	90.10	90.79
159	372	16	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	86.63	84.66	84.54	83.76	86.59	85.53	86.50	86.99	87.35	88.42	87.92	90.59	88.28	86.22	87.39	88.12	87.10	88.33	86.72	93.02	89.38	90.40	87.25	90.93	90.07
336	372	16	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	87.64	61.30	87.87	60.32	90.47	91.06	89.66	59.74	90.93	59.59	92.06	89.31	87.80	58.86	59.21	89.18	58.78	91.43	89.09	91.96	58.79	58.44	59.20	59.65	57.92
348	372	16	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	89.68	61.30	88.66	60.53	89.72	93.22	89.91	60.42	88.25	90.94	58.35	93.18	87.08	91.68	90.29	91.42	59.61	93.22	89.16	92.17	55.85	57.59	92.46	57.11	57.41
351	372	16	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	87.64	59.90	87.64	61.82	89.67	89.14	88.40	89.41	89.93	89.90	91.80	89.03	91.65	87.67	43.72	86.68	90.92	90.42	57.16	89.15	88.62	56.83	58.84	59.35	60.03
528	372	16	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	90.17	90.92	59.26	90.49	88.41	92.18	89.90	90.13	90.71	89.42	92.47	56.56	87.56	60.02	58.79	90.37	59.55	92.83	57.08	58.76	58.85	56.76	56.88	58.04	56.86
531	372	16	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	88.23	88.67	88.41	90.09	89.81	89.27	90.29	59.27	88.25	89.66	89.78	58.16	90.36	90.97	60.61	90.91	59.27	91.15	92.44	90.37	89.38	57.79	58.88	57.38	58.39

Size of the All data:  (100, 28)
Size of the Sig data:  (46, 33)

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params_mw1 = dt_valid.loc[dt_valid_sub_mw1.index][['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
optimal_params_mw = optimal_params_mw1.reset_index()
optimal_params_mw.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_mw1 = pd.merge(optimal_params_mw, dt_test, on=['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg'])
test_data_1_sub_mw = test_data_mw1[list(test_data_mw1.columns[test_data_mw1.columns.str.startswith('mAP_test')])]

Equivalent test data

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_abs_values	mAP_test_abs_values_2	mAP_test_abs_values_3	mAP_test_abs_values_4	mAP_test_abs_values_5	mAP_test_abs_values_6	mAP_test_abs_values_7	mAP_test_abs_values_8	mAP_test_abs_values_9	mAP_test_abs_values_10	mAP_test_abs_values_11	mAP_test_abs_values_12	mAP_test_abs_values_13	mAP_test_abs_values_14	mAP_test_abs_values_15	mAP_test_abs_values_16	mAP_test_abs_values_17	mAP_test_abs_values_18	mAP_test_abs_values_19	mAP_test_abs_values_20	mAP_test_abs_values_21	mAP_test_abs_values_22	mAP_test_abs_values_23	mAP_test_abs_values_25	mAP_test_abs_values_26
17	372	16	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	91.42	89.71	93.65	87.12	90.62	91.85	94.29	90.54	89.90	91.12	92.60	88.77	89.97	90.32	59.15	89.51	59.15	91.24	85.68	87.29	91.10	90.42	90.18	91.86	59.31
8	372	16	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	92.35	90.96	94.04	89.83	88.15	91.19	93.21	91.17	93.41	89.31	93.20	87.84	94.32	92.77	70.79	94.44	91.10	90.53	90.15	91.43	83.08	92.39	93.26	91.48	90.78
1	372	16	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	90.95	86.68	93.58	67.88	93.04	92.25	93.49	91.38	90.20	94.45	92.70	89.92	91.61	92.15	89.66	92.60	95.88	94.86	94.65	91.40	93.40	93.46	93.06	59.93	91.23
14	372	16	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	91.80	92.49	93.06	92.59	94.92	93.93	93.16	93.44	89.70	93.43	91.94	91.10	90.16	91.81	90.04	93.73	92.41	91.51	93.67	91.80	92.76	86.25	92.01	91.46	91.58
2	372	16	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	94.20	93.47	93.99	92.52	94.83	95.10	91.75	93.41	95.11	94.35	92.90	91.74	89.98	89.75	93.46	93.76	95.16	92.90	93.48	93.37	91.10	92.29	93.46	91.12	88.63
37	372	16	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	58.84	61.05	61.47	59.25	61.92	59.43	58.95	61.83	58.57	61.42	59.62	61.93	59.42	59.83	61.41	59.28	90.43	58.21	58.16	58.63	59.16	62.87	62.67	62.50	61.72
44	372	16	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	59.49	61.87	59.93	61.72	61.94	60.61	59.14	62.14	58.97	61.99	59.25	59.89	59.67	59.52	60.57	59.87	62.28	59.01	59.48	61.90	62.32	62.55	50.39	61.94	62.03
36	372	16	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	59.18	62.08	59.37	59.20	59.79	59.92	59.14	61.71	62.06	59.41	59.90	59.58	59.99	58.74	61.76	59.68	61.56	59.33	62.64	59.88	62.45	62.55	61.60	62.46	61.76
29	372	16	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	58.99	59.72	59.33	61.59	59.20	59.78	59.23	62.15	58.61	61.98	60.84	59.90	59.14	61.95	62.06	59.43	59.84	58.74	51.80	59.95	61.56	62.22	61.65	62.39	62.03
11	372	16	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	92.95	91.59	94.03	92.10	93.67	91.16	93.56	92.03	92.80	94.10	91.70	93.22	90.51	93.23	85.61	93.76	91.67	94.15	91.63	93.87	94.04	92.72	91.05	59.53	92.98
4	372	16	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	90.08	89.78	95.16	91.27	94.40	92.24	93.73	59.22	89.69	94.23	93.56	90.63	95.43	92.18	93.14	93.92	91.36	92.30	92.24	94.37	93.13	92.95	91.65	60.70	92.84
10	372	16	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	94.85	93.37	94.58	93.60	88.49	92.41	95.60	91.14	93.18	93.32	93.02	91.45	94.45	94.91	93.96	90.87	90.43	93.69	92.31	89.46	90.15	58.98	93.92	92.34	91.91
34	372	16	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	59.79	61.28	58.64	61.99	61.93	59.98	58.63	59.17	58.51	59.12	59.33	62.32	59.43	59.56	61.65	59.57	60.29	58.74	59.80	59.43	62.48	61.56	62.34	62.99	61.94
28	372	16	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	59.99	61.99	61.65	61.93	59.41	59.71	59.19	59.84	58.19	58.54	60.06	62.70	59.14	59.61	61.48	59.13	62.56	59.64	60.15	58.56	60.63	61.83	59.88	62.78	62.10
38	372	16	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	59.39	59.15	59.57	62.39	60.27	60.02	59.38	62.32	59.42	59.56	62.97	59.32	59.63	60.72	61.93	59.58	61.70	59.52	61.93	50.96	62.34	62.57	87.88	62.15	62.13
26	372	16	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	59.62	61.93	50.73	61.93	59.34	59.86	59.41	59.86	52.41	59.37	59.68	59.80	59.67	58.37	61.11	60.27	58.97	61.45	59.96	61.99	61.22	61.92	59.43	62.46	61.81
20	372	16	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	59.47	49.98	58.86	60.19	60.23	59.78	57.95	51.19	59.11	59.57	60.38	60.49	52.74	60.08	59.35	58.63	58.51	59.69	59.90	59.87	62.55	60.78	59.69	61.83	62.45
0	372	16	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	94.19	90.90	90.84	91.66	95.26	91.57	94.89	93.40	94.35	93.48	92.71	92.47	92.06	94.79	93.24	95.37	90.54	91.11	92.75	92.35	58.52	92.53	91.17	92.05	93.54
15	372	16	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	94.28	90.49	93.53	91.49	93.49	92.08	94.27	93.27	92.27	91.95	93.36	92.92	91.38	94.82	87.56	93.06	93.74	93.11	92.68	94.12	92.64	95.06	91.86	52.49	59.55
16	372	16	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	95.06	89.72	91.66	93.80	92.61	91.53	93.76	93.01	92.28	92.95	91.69	93.94	94.12	91.35	94.42	93.33	96.19	92.82	92.79	92.19	93.98	93.16	94.17	92.58	68.98
35	372	16	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	59.56	50.77	59.58	61.70	61.99	59.85	59.30	62.70	59.29	61.99	59.71	59.47	59.91	58.97	61.07	59.24	48.63	58.33	59.95	59.65	61.43	62.59	60.43	62.79	61.67
30	372	16	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	58.39	59.87	59.27	62.34	58.04	59.76	59.26	60.81	59.23	59.02	60.27	61.99	59.34	58.97	59.44	88.21	61.26	59.34	62.44	59.67	62.48	61.76	61.64	62.04	62.83
25	372	16	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	59.23	60.81	59.32	62.32	59.43	59.79	59.22	61.54	50.51	58.64	59.95	61.59	59.54	60.61	59.34	59.42	59.30	89.36	59.52	58.53	60.79	63.09	60.02	63.04	63.27
7	372	16	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	94.05	91.38	94.72	93.19	90.28	93.68	93.75	92.66	92.64	94.20	93.23	94.81	94.08	92.23	91.67	92.96	92.14	94.84	94.51	92.27	92.53	91.61	89.13	61.90	87.80
9	372	16	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	93.36	92.41	92.03	90.34	90.39	93.05	94.18	93.77	92.96	93.39	91.64	93.58	92.92	95.46	93.51	91.97	90.96	92.04	92.89	93.30	91.80	95.07	88.88	93.09	92.51
41	372	16	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	60.04	61.75	59.80	59.81	61.99	50.27	59.95	61.77	59.55	59.34	60.77	88.17	59.69	62.41	62.64	90.69	60.96	58.88	60.43	62.26	62.46	63.04	59.57	62.34	62.61
31	372	16	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	93.25	62.32	62.64	61.79	59.80	59.18	59.69	60.13	59.70	59.57	61.20	61.07	59.33	59.91	60.90	59.71	61.32	59.57	60.22	60.32	62.55	63.11	62.72	61.64	62.26
45	372	16	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	59.14	60.32	59.82	62.64	59.19	60.37	59.53	61.76	51.03	59.61	60.67	61.26	60.80	60.72	62.64	59.23	60.95	59.56	62.97	62.32	62.72	61.39	62.17	62.80	61.33
39	372	16	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	59.52	59.79	59.14	59.95	59.80	60.87	52.25	62.97	59.37	62.03	62.97	62.39	60.97	61.21	62.32	59.61	61.70	59.71	59.80	59.62	60.25	62.75	62.48	62.72	62.03
6	372	16	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	92.67	94.38	93.26	91.17	94.49	92.91	91.39	89.88	93.59	93.35	94.72	93.54	94.80	88.14	93.49	91.57	91.68	93.77	91.13	92.37	61.03	93.21	94.43	93.94	89.94
13	372	16	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	95.16	92.56	88.22	90.20	91.91	92.76	94.47	94.19	92.58	92.09	92.14	91.85	92.77	94.70	51.18	94.56	94.38	92.64	92.37	93.06	61.67	92.36	91.43	94.84	92.74
5	372	16	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	95.00	93.69	95.54	91.07	89.48	92.31	93.35	92.64	92.31	93.84	89.18	94.39	90.38	92.86	91.41	93.57	90.27	89.99	92.21	92.87	92.24	92.54	61.11	92.86	70.78
33	372	16	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	59.91	62.06	59.85	60.22	59.62	59.19	92.87	60.03	60.08	90.63	50.51	61.71	59.62	59.21	62.06	60.37	59.90	59.58	60.27	61.99	62.25	62.70	62.39	61.78	62.75
40	372	16	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	59.45	59.70	61.74	59.72	59.21	60.33	61.20	62.41	59.81	59.50	51.47	62.72	60.34	58.92	61.87	59.85	61.61	59.98	60.22	62.32	61.55	62.45	62.79	62.50	61.81
27	372	16	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	59.42	52.56	59.93	62.39	59.46	59.33	60.53	50.74	60.65	59.57	62.97	61.93	51.36	60.72	61.76	50.40	61.66	60.86	60.83	59.53	62.36	62.57	60.44	63.09	62.44
32	372	16	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	60.43	59.65	59.41	61.99	59.81	59.45	60.40	62.90	59.46	61.76	60.73	62.32	60.60	59.86	52.81	59.54	61.64	59.61	60.77	60.77	62.48	62.39	62.60	62.32	62.97
22	372	16	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	60.32	61.01	59.62	59.48	59.80	60.77	59.48	62.26	60.77	60.89	49.99	60.00	59.57	60.03	61.45	91.04	59.71	60.81	60.43	61.94	62.91	62.03	62.71	62.68	51.89
18	372	16	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	94.36	92.56	94.32	91.82	95.50	90.05	92.67	93.12	94.56	93.18	92.63	92.37	94.68	94.64	88.37	92.70	48.12	93.22	89.68	91.30	91.74	92.36	92.46	56.53	92.61
19	372	16	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	94.33	96.37	91.19	93.37	95.37	92.15	95.78	93.31	93.31	93.08	93.84	92.27	91.78	87.67	48.12	94.38	48.12	90.83	92.11	92.25	93.78	93.33	93.26	91.47	91.64
12	372	16	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	90.71	94.11	90.14	92.09	94.61	92.87	93.48	91.67	94.93	94.47	92.71	92.29	95.04	92.02	94.50	94.52	93.27	94.86	92.32	92.34	93.98	94.02	94.16	92.60	90.30
3	372	16	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	92.55	88.17	93.56	87.90	93.08	93.26	92.77	93.30	94.07	93.37	91.79	93.07	91.98	95.89	94.92	93.92	92.10	92.87	93.10	91.17	93.96	92.72	92.46	90.35	93.19
42	372	16	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	60.37	61.99	60.27	61.93	59.31	59.63	59.95	62.88	59.57	61.76	60.22	60.94	61.31	61.03	61.50	51.78	61.61	59.56	61.13	59.16	61.45	62.35	62.26	62.32	62.44
23	372	16	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	59.86	63.04	59.77	61.99	59.31	59.76	60.29	62.26	91.13	59.79	62.64	60.11	50.28	59.85	59.95	59.86	62.09	59.23	51.59	59.99	62.55	62.34	59.05	62.43	62.30
24	372	16	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	59.85	62.64	60.19	62.06	59.95	60.77	59.53	59.72	59.83	60.09	59.90	61.20	59.62	89.99	48.12	51.38	59.66	59.65	62.64	60.67	60.26	63.04	62.79	62.77	61.68
43	372	16	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	59.56	59.80	62.32	59.50	60.12	60.01	60.08	60.50	59.69	59.86	59.71	62.84	60.73	61.53	61.70	60.53	61.24	59.58	62.97	62.06	62.66	63.04	62.57	62.03	62.53
21	372	16	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	59.95	60.17	59.47	59.36	59.57	61.30	60.00	61.52	60.83	60.03	61.07	62.41	60.29	60.08	61.48	60.33	61.01	60.27	59.80	88.26	60.26	63.02	62.39	62.36	61.87

Size of the test data:  (46, 33)

Difference

diff_mw = dt_mw[['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
for i, cols in enumerate(zip(dt_mw.columns,test_data_mw1.columns)):   
   diff_mw[f'col_diff_{i}'] = (np.array(dt_mw[cols[0]]) - np.array(test_data_mw1[cols[1]]))*-1

# Function to count negatives in a row
def count_negatives(row):
    return sum(1 for value in row if isinstance(value, (int, float)) and value < 0)

diff_mw['negative_count'] = diff_mw.apply(count_negatives, axis=1)

diff_mw.sort_values(by =["negative_count"],inplace = True, ascending=False)


html_table = diff_mw.to_html(index=True)

# Wrap in a scrollable div
scrollable_table = f"""
<div style="height: 400px; width: 100%; overflow-x: auto; overflow-y: auto;">
    {html_table}
</div>
"""
# Display the scrollable table
display(HTML(scrollable_table))

key_values = ['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']
print('Are the keys of the valid and test dfs same?: ',dt_mw[key_values].equals(test_data_mw1[key_values]))

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	col_diff_0	col_diff_1	col_diff_2	col_diff_3	col_diff_4	col_diff_5	col_diff_6	col_diff_7	col_diff_8	col_diff_9	col_diff_10	col_diff_11	col_diff_12	col_diff_13	col_diff_14	col_diff_15	col_diff_16	col_diff_17	col_diff_18	col_diff_19	col_diff_20	col_diff_21	col_diff_22	col_diff_23	col_diff_24	col_diff_25	col_diff_26	col_diff_27	col_diff_28	col_diff_29	col_diff_30	col_diff_31	col_diff_32	negative_count
1052	372	16	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.77	-6.31	-30.32	-29.70	-29.84	-32.42	-30.44	-35.67	-31.40	-31.82	-31.29	-30.91	-33.22	-31.61	-32.40	-33.10	-33.82	-31.45	-32.02	-33.43	5.21	0.58	-31.22	2.30	3.84	21
284	372	16	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.56	-5.71	-30.53	-0.48	-0.71	-30.58	-30.16	3.14	-32.05	0.92	-32.26	-33.23	-29.85	-31.07	-0.79	-31.95	-6.68	-32.70	-31.48	-32.70	1.54	4.30	-29.49	3.72	2.84	19
531	372	16	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.28	-28.50	-28.94	-30.73	-30.24	-27.97	-30.29	2.25	-27.42	-29.63	-28.71	4.25	-30.07	-30.89	0.87	-30.58	1.74	-30.88	-32.64	-2.11	-29.12	5.23	3.51	4.98	3.48	17
1119	372	16	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.07	-0.62	-29.80	-31.24	-31.13	-29.85	-29.44	1.82	-28.59	-28.15	-39.75	-31.42	-32.10	-30.14	2.41	3.91	-32.46	-27.57	-30.99	4.17	6.46	4.29	4.87	5.82	-35.61	17
479	372	16	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.72	-1.06	-30.90	1.03	-29.54	-32.25	-32.15	1.76	-6.01	-32.07	-32.49	4.01	-30.97	1.69	-30.11	-32.51	-32.43	5.46	-32.95	-31.63	-27.93	7.19	-29.63	5.07	7.02	17
1043	372	16	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.12	0.39	-5.29	1.17	-30.89	-31.69	-31.42	-30.07	-35.62	-31.11	-33.26	-32.41	-32.50	-31.76	0.41	-29.89	-32.26	-0.06	-31.72	3.00	0.85	2.54	-31.88	3.14	2.17	17
1436	372	16	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.25	0.75	-29.07	-29.60	-28.76	-30.00	-28.30	-0.24	-0.31	-32.03	-31.24	-30.87	-27.66	-32.72	1.23	-30.74	0.13	-31.85	4.29	-31.41	4.03	4.01	-0.31	0.91	2.09	17
927	372	16	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.04	-34.53	-29.98	1.43	-29.76	-32.64	-25.30	-36.64	-29.70	-31.64	4.45	1.68	-35.91	-28.38	1.45	-38.32	1.96	-27.81	-29.80	-31.64	3.79	4.91	-27.61	5.39	2.58	16
848	372	16	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.14	0.59	-30.47	-0.32	0.64	-32.41	-30.86	-31.37	-31.50	-31.83	-32.57	2.89	-31.12	-32.16	1.31	-30.01	-0.34	-32.13	-31.26	-31.90	5.21	1.51	2.50	4.30	3.46	16
851	372	16	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.06	0.34	-0.07	0.88	-29.79	-32.72	-30.09	-31.14	-32.72	-31.68	-31.99	3.82	-32.82	-31.00	3.03	-33.84	3.72	-30.07	-31.52	-33.90	2.59	2.88	-30.37	4.71	4.49	16
351	372	16	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.79	2.74	-27.45	0.24	-29.72	-28.37	-28.87	-29.69	-30.10	-29.81	-31.90	-27.83	-32.03	2.32	4.40	-35.30	-31.26	-30.77	5.48	-28.48	-28.36	6.21	3.95	3.42	1.65	16
1107	372	16	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-26.96	-30.04	-29.31	0.92	-30.38	-32.77	-26.45	3.65	-31.23	2.21	-29.52	4.27	-29.75	-32.06	-35.96	-31.73	1.34	-30.81	-29.85	-29.86	5.42	4.92	4.93	2.65	5.86	15
1439	372	16	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.48	-30.10	-29.46	2.58	-31.99	-31.88	-31.98	1.33	-32.13	3.18	-29.42	-32.28	-31.82	2.69	1.27	-32.12	-30.98	-33.36	-37.26	-32.98	0.80	4.24	2.17	4.57	3.88	15
1427	372	16	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.18	-0.21	-27.10	0.26	0.68	-30.01	-29.44	1.47	-31.57	0.39	-32.21	-32.32	-31.25	-31.95	-2.45	-31.27	1.55	-32.15	-33.49	3.28	2.98	3.48	-6.55	1.89	2.77	15
348	372	16	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.82	1.74	-28.89	1.46	-30.41	-33.46	-29.62	1.84	2.88	-31.15	4.29	-33.07	-36.80	-31.83	-30.34	-31.56	2.48	-33.99	-37.57	-32.18	6.70	4.75	-33.41	5.32	4.89	15
924	372	16	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.75	-28.98	-1.73	-28.96	-29.99	-30.57	-24.59	3.26	-29.60	-31.21	-39.93	6.39	-29.27	-31.79	1.60	-31.34	1.95	-29.68	-30.44	3.15	5.32	6.11	5.51	6.12	2.61	15
912	372	16	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.75	-0.16	-27.83	-28.93	-29.98	-33.52	6.06	-29.38	-30.33	8.06	-5.73	2.50	-31.81	-32.98	2.28	-30.54	-31.02	-32.14	-32.42	3.47	6.60	6.00	4.96	2.53	5.67	15
1503	372	16	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.91	-29.38	-29.01	-29.46	-29.52	-29.99	-33.76	4.18	-31.17	2.83	6.10	2.96	-27.30	-0.11	2.84	-32.57	2.19	-30.73	-33.67	-32.85	-29.52	4.57	4.92	5.99	3.64	15
1491	372	16	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.56	1.15	2.84	1.53	-28.37	-32.79	-29.72	-31.02	-32.25	-29.85	-27.59	-28.73	-31.12	-31.01	1.60	-31.72	3.22	-31.10	-30.97	-30.60	4.49	7.87	4.20	2.69	3.03	14
287	372	16	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.00	-30.78	-29.58	1.15	-30.59	-32.46	-30.35	1.13	-32.31	-32.47	-30.78	3.20	-31.16	-32.48	-33.21	5.26	2.11	-31.28	3.51	-31.52	5.59	3.62	2.24	2.04	5.26	14
336	372	16	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.27	0.69	-27.60	1.61	-31.16	-31.43	-29.71	3.14	-31.36	2.17	-31.84	-28.37	-26.49	2.17	2.29	-37.40	2.83	-31.87	-27.96	-32.80	2.66	3.91	3.06	2.67	4.52	13
863	372	16	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.52	-30.51	-28.35	1.73	-29.08	-29.68	-30.44	1.60	-31.92	-32.16	6.19	-34.40	-30.36	1.17	1.88	-31.63	0.83	-31.92	1.68	-37.74	4.21	4.93	4.80	1.65	3.78	13
1424	372	16	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.59	1.06	0.61	-31.06	1.41	-32.54	-30.52	1.34	-30.61	1.26	-32.85	4.16	-33.22	-31.27	1.08	-33.80	6.47	-32.58	-33.40	-32.32	-32.38	4.17	5.04	4.27	2.32	13
528	372	16	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.61	-31.12	3.06	-30.99	-28.29	-32.17	-29.82	-29.63	-31.02	-29.56	-32.76	6.28	-26.83	1.51	2.91	-29.84	1.69	-33.25	5.89	3.30	3.81	6.28	5.69	3.99	5.67	13
1500	372	16	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-31.56	-29.33	-28.58	2.22	-29.77	-31.27	-30.42	1.85	-36.58	-30.83	-29.46	-27.03	-27.24	1.08	2.28	-32.99	2.30	-31.12	6.46	3.32	6.57	2.35	4.89	6.63	2.05	13
1488	372	16	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.10	0.48	-28.37	-30.12	0.94	-37.05	-30.21	1.13	-32.65	-29.62	-28.84	6.82	-30.47	1.55	3.69	4.98	2.96	-33.83	-29.71	5.04	5.38	6.43	-30.29	5.04	6.73	12
607	372	16	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	23.92	20.43	5.46	-1.60	1.88	6.28	10.38	-0.25	-0.55	0.80	3.72	23.76	1.99	0.01	-25.31	5.57	-28.39	4.81	2.75	0.30	0.36	2.22	2.03	2.72	-28.92	6
735	372	16	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	10.70	7.77	10.91	6.17	6.51	4.02	8.99	5.11	5.67	6.06	-1.50	6.54	0.64	6.35	2.93	7.69	-0.63	4.91	2.97	3.30	2.12	2.86	-27.47	2.36	-20.54	4
1232	372	16	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.01	2.46	9.04	4.72	2.61	1.76	8.17	0.81	8.92	1.91	6.13	-0.72	8.86	3.81	-18.06	6.79	4.03	3.07	2.10	1.15	-6.81	2.15	6.20	2.60	2.05	3
1235	372	16	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	1.90	-2.05	9.14	-19.79	4.39	3.78	7.97	5.20	5.03	5.82	4.55	2.94	8.59	4.34	1.59	5.08	8.21	6.59	5.71	2.57	4.93	3.69	4.10	-29.23	1.59	3
723	372	16	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.93	3.26	1.59	2.55	5.34	1.25	10.44	5.06	3.58	5.08	1.15	4.10	5.48	6.54	-38.04	7.79	6.80	3.87	3.55	-0.90	-29.48	3.02	3.88	38.71	2.72	3
1244	372	16	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	4.71	36.23	2.67	5.03	8.74	6.80	8.06	6.64	4.86	6.38	2.52	1.63	3.64	4.41	-0.46	4.16	1.73	3.83	5.78	-0.54	5.27	-0.91	2.12	2.34	3.42	3
92	372	16	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.62	34.88	7.32	4.42	10.61	5.73	10.41	4.40	1.58	7.56	5.48	0.43	3.60	9.33	-3.29	5.15	4.71	2.79	3.74	5.47	4.78	6.60	2.05	-37.42	-30.22	3
668	372	16	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	4.49	3.69	9.10	5.09	10.20	4.85	9.93	-29.99	2.85	7.76	3.26	4.37	8.88	5.87	4.39	5.06	1.75	5.47	1.48	6.13	3.68	3.63	2.57	-30.33	4.41	2
659	372	16	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.43	5.04	8.65	7.47	11.25	5.64	9.58	5.31	7.40	3.60	5.36	3.45	1.88	2.21	-0.86	6.43	5.00	7.19	2.34	2.05	5.70	2.91	2.24	-31.62	4.62	2
159	372	16	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.92	3.51	9.02	4.14	6.49	7.73	6.27	6.31	6.72	4.95	3.87	2.48	3.70	9.67	7.53	5.80	5.00	4.54	6.38	-1.85	4.58	2.32	5.21	-0.58	3.12	2
1308	372	16	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.54	6.98	8.28	5.52	5.65	4.08	8.32	5.85	5.12	5.35	0.88	8.94	5.98	6.86	6.67	3.80	3.47	9.07	5.44	4.59	4.70	3.72	2.16	-33.01	-1.80	2
144	372	16	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	10.04	7.77	7.83	5.20	10.26	1.44	6.71	4.91	7.37	5.57	4.18	3.72	7.75	6.70	0.07	37.24	4.40	4.95	1.94	2.26	2.82	3.85	3.12	-35.22	3.29	1
156	372	16	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	6.68	8.62	5.52	7.98	5.61	3.79	7.60	7.03	9.79	9.16	1.12	2.98	9.03	5.14	6.55	6.47	6.12	7.50	4.05	4.74	4.96	5.02	4.69	2.50	-0.49	1
671	372	16	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	11.59	7.14	7.66	6.93	1.47	3.45	11.66	3.18	7.50	5.88	6.03	2.62	8.43	6.50	6.52	1.98	3.99	6.58	1.89	2.00	2.39	-30.86	3.28	4.02	3.76	1
83	372	16	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.05	5.80	3.63	5.65	9.00	4.93	7.59	6.72	5.77	7.97	3.43	7.00	3.04	6.33	3.75	7.08	2.09	1.30	2.63	4.22	-31.09	3.63	1.36	2.71	4.34	1
1247	372	16	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.62	8.88	7.67	7.03	8.36	7.31	6.24	6.60	7.59	6.56	2.73	0.85	4.94	1.90	8.27	6.96	6.61	6.80	2.37	2.37	4.19	2.22	4.35	1.66	-3.11	1
95	372	16	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	10.39	3.77	7.45	4.33	6.84	1.25	6.52	4.51	5.53	5.43	7.17	40.43	8.19	4.75	4.54	5.53	10.14	5.95	4.24	2.57	6.32	4.40	5.10	4.61	-22.68	1
720	372	16	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.54	7.81	9.43	3.57	10.15	6.27	4.85	3.98	5.01	8.38	7.19	5.11	7.37	1.25	5.84	5.04	1.33	5.24	0.39	1.00	-33.42	4.96	5.95	5.19	3.06	1
1311	372	16	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.50	4.11	5.82	6.22	5.79	7.02	8.76	5.43	6.83	6.66	-0.11	5.11	4.77	9.04	5.64	4.30	4.65	4.27	1.03	5.35	2.76	6.04	1.16	4.75	2.29	1
147	372	16	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.95	10.70	7.35	3.59	8.88	7.32	8.89	5.37	7.47	4.32	6.29	0.58	4.61	0.61	4.40	7.87	4.40	1.76	2.53	1.81	4.91	4.43	4.64	1.06	2.86	0

Are the keys of the valid and test dfs same?:  False

Model performance on valid data

Average & Std Deviation of the Significant rows:

	mean	std
mAP_valid_abs_values_6	89.692609	2.358115
mAP_valid_abs_values_18	88.709565	4.639103
mAP_valid_abs_values_13	88.615652	2.176657
mAP_valid_abs_values_16	88.390000	5.496899
mAP_valid_abs_values_9	87.465652	6.439106
mAP_valid_abs_values_7	87.370652	2.312300
mAP_valid_abs_values	86.772609	3.547403
mAP_valid_abs_values_11	86.520652	10.473764
mAP_valid_abs_values_19	85.927174	11.081484
mAP_valid_abs_values_20	84.903043	12.363281
mAP_valid_abs_values_5	84.809348	8.580876
mAP_valid_abs_values_10	84.328261	10.627476
mAP_valid_abs_values_14	83.989130	11.388991
mAP_valid_abs_values_3	83.725435	9.420535
mAP_valid_abs_values_12	80.076957	14.657660
mAP_valid_abs_values_4	78.338043	12.957241
mAP_valid_abs_values_8	77.857391	13.839314
mAP_valid_abs_values_23	77.661739	15.001737
mAP_valid_abs_values_2	76.363043	13.577547
mAP_valid_abs_values_17	75.335435	15.988766
mAP_valid_abs_values_21	74.916957	15.871967
mAP_valid_abs_values_15	74.224565	15.543830
mAP_valid_abs_values_26	72.500652	15.670258
mAP_valid_abs_values_22	71.451304	15.665067
mAP_valid_abs_values_25	71.448696	15.766411

Model performance on Test data

We then apply the hyperparameters to the test set and average the results.

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params = dt_valid.loc[dt_valid_sub_mw1.index].iloc[:, :6].reset_index()
optimal_params.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_1 = pd.merge(optimal_params, dt_test, how='left')
test_data_1_sub = test_data_1[list(test_data_1.columns[test_data_1.columns.str.startswith('mAP_test')])]
test_data_1['average_map'] = test_data_1_sub.apply(np.mean, axis=1)

test_data_1.sort_values(by=['average_map'], ascending= False,inplace=True)

	mean	std
mAP_test_abs_values_16	75.233804	17.072799
mAP_test_abs_values_10	75.211739	16.358892
mAP_test_abs_values_7	75.181304	17.599944
mAP_test_abs_values_12	75.113641	15.163913
mAP_test_abs_values	74.867120	16.746625
mAP_test_abs_values_20	74.587663	15.959395
mAP_test_abs_values_22	74.485652	14.786154
mAP_test_abs_values_5	74.429239	16.417018
mAP_test_abs_values_9	74.390761	17.139923
mAP_test_abs_values_13	74.256793	16.781006
mAP_test_abs_values_3	74.243967	16.495746
mAP_test_abs_values_23	74.239402	15.938178
mAP_test_abs_values_11	74.125924	16.422341
mAP_test_abs_values_14	73.965326	16.245739
mAP_test_abs_values_19	73.934891	16.194125
average_map	73.916957	14.846075
mAP_test_abs_values_8	73.874239	16.010211
mAP_test_abs_values_6	73.796630	16.320558
mAP_test_abs_values_2	73.760924	15.701428
mAP_test_abs_values_18	73.760815	16.664212
mAP_test_abs_values_26	73.225380	14.351369
mAP_test_abs_values_17	72.906630	16.441557
mAP_test_abs_values_21	72.857011	15.007684
mAP_test_abs_values_4	72.852609	15.409223
mAP_test_abs_values_25	71.477554	14.252064
mAP_test_abs_values_15	71.144891	15.373538

Summary using radar plot

def extract_number(text):
    if isinstance(text, str):
        matches = re.findall(r'\d+', text)
        return int(matches[0]) if matches else 1
    return 1

res_valid1['id'] = res_valid1.index.to_series().apply(extract_number)
res_test1['id'] = res_test1.index.to_series().apply(extract_number)



res_comb = pd.concat([res_valid1,res_test1])
index_series = res_comb.index.to_series()

res_comb['type'] = np.where(
    index_series.str.contains('valid', case=False), 'valid',
    np.where(index_series.str.contains('test', case=False), 'test', 'unknown')
)
res_comb = res_comb.query("type !='unknown'")

res_comb =  res_comb.reset_index(drop=True)


res_test1 = res_test1.sort_values(by=['id']).reset_index().query("index !='average_map'")
data_range1 = np.array(list(res_test1['mean']) + list(res_valid1['mean']))
categories = [str(i) for i in range(1,26)]
fig = go.Figure()

# Valid
fig.add_trace(go.Scatterpolar(
       r=list(res_test1['mean']),
      theta=categories,
      #fill='toself',
      name='Test'
))

# Test
fig.add_trace(go.Scatterpolar(
      r=list(res_valid1['mean']),
      theta=categories,
      #fill='toself',
      name='Valid'
))

# Customization of chart
fig.update_layout(
  title={
      'text': "T-Test",
        'xanchor': 'center',
      'yanchor': 'top'
  },
  polar=dict(
    radialaxis=dict(
      visible=True,
      range=[round(data_range1.min()), round(data_range1.max()) + 1]
    )),
  showlegend=True
)

fig.show()

##############


res_valid2['id'] = res_valid2.index.to_series().apply(extract_number)
res_test2['id'] = res_test2.index.to_series().apply(extract_number)



res_comb = pd.concat([res_valid2,res_test2])
index_series = res_comb.index.to_series()

res_comb['type'] = np.where(
    index_series.str.contains('valid', case=False), 'valid',
    np.where(index_series.str.contains('test', case=False), 'test', 'unknown')
)
res_comb = res_comb.query("type !='unknown'")

res_comb =  res_comb.reset_index(drop=True)


res_test2 = res_test2.sort_values(by=['id']).reset_index().query("index !='average_map'")

data_range2 = np.array(list(res_test2['mean']) + list(res_valid2['mean']))

categories = [str(i) for i in range(1,26)]
fig = go.Figure()

# Valid
fig.add_trace(go.Scatterpolar(
       r=list(res_test2['mean']),
      theta=categories,
      #fill='toself',
      name='Test'
))

# Test
fig.add_trace(go.Scatterpolar(
      r=list(res_valid2['mean']),
      theta=categories,
      #fill='toself',
      name='Valid'
))

# Customization of chart
fig.update_layout(
  title={
      'text': "MU-Test",
         'xanchor': 'center',
      'yanchor': 'top'
  },
  polar=dict(
    radialaxis=dict(
      visible=True,
      range=[round(data_range2.min()), round(data_range2.max()) + 1]
    )),
  showlegend=True
)

fig.show()

Experiment 2 Results

• In these experimental results, the thresholding is based on a fixed value of 0. This decision is informed by the fact that the binary-like hash values are symmetrically distributed between -1 and 1.

Validation Data mAP Results Preview:

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_zero	mAP_valid_zero_2	mAP_valid_zero_3	mAP_valid_zero_4	mAP_valid_zero_5	mAP_valid_zero_6	mAP_valid_zero_7	mAP_valid_zero_8	mAP_valid_zero_9	mAP_valid_zero_10	mAP_valid_zero_11	mAP_valid_zero_12	mAP_valid_zero_13	mAP_valid_zero_14	mAP_valid_zero_15	mAP_valid_zero_16	mAP_valid_zero_17	mAP_valid_zero_18	mAP_valid_zero_19	mAP_valid_zero_20	mAP_valid_zero_21	mAP_valid_zero_22	mAP_valid_zero_23	mAP_valid_zero_25	mAP_valid_zero_26
16	372	16	0.1	16	0.00100	24	0.000000e+00	0.000000e+00	83.15	86.55	67.73	85.60	85.90	85.94	82.33	81.63	80.29	87.24	86.82	88.15	88.12	59.43	81.64	82.15	85.18	85.32	78.90	86.64	87.54	79.08	86.53	65.31	85.57
19	372	16	0.1	16	0.00100	24	0.000000e+00	1.000000e-08	82.86	86.57	84.84	71.42	82.57	84.01	82.80	86.21	86.18	84.43	85.14	86.11	84.22	85.29	87.03	87.00	88.05	85.17	88.14	88.24	87.10	85.35	83.32	88.04	86.10
28	372	16	0.1	16	0.00100	24	1.000000e-08	0.000000e+00	81.77	85.53	84.19	89.00	85.19	84.56	84.87	86.17	89.47	83.70	85.83	86.68	86.09	89.30	84.68	82.99	84.55	73.29	88.15	79.98	89.28	87.93	86.33	89.59	87.58
31	372	16	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	83.04	83.53	83.89	86.76	83.84	83.60	83.96	80.74	84.88	86.16	88.51	88.70	84.78	87.61	88.08	82.76	43.72	86.56	88.95	84.07	84.59	86.17	86.12	87.55	85.72
80	372	16	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	84.63	88.42	83.59	84.03	87.12	84.90	84.35	87.86	87.60	85.67	90.03	84.55	87.08	43.72	90.41	86.71	86.93	88.59	86.21	87.19	89.49	88.83	86.98	89.67	86.19
83	372	16	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	85.78	85.10	87.06	85.85	85.75	86.08	87.05	86.50	88.49	85.51	89.27	85.33	88.33	88.43	89.13	88.14	87.98	89.70	90.01	87.96	89.56	88.86	89.76	89.34	88.81
92	372	16	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	85.33	55.49	85.95	86.99	82.75	86.35	83.79	88.80	90.37	83.49	87.88	92.20	87.57	85.43	90.77	87.66	89.01	89.83	88.83	88.65	87.71	88.24	89.60	89.37	89.74
95	372	16	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	84.39	85.29	84.06	88.72	85.74	90.04	87.22	87.87	86.75	87.21	84.39	52.60	85.86	86.53	89.72	87.72	84.87	86.44	88.54	89.61	87.53	88.71	88.63	87.39	91.23
144	372	16	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	83.67	84.79	86.13	86.58	85.21	88.42	85.83	88.01	87.18	87.61	88.45	88.65	86.86	87.87	88.30	55.25	43.72	88.15	87.41	88.98	88.66	88.26	88.39	91.72	89.32
147	372	16	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	86.13	85.61	83.84	89.78	86.49	84.58	86.42	87.88	84.90	88.74	87.54	91.68	87.17	86.77	43.72	86.43	43.72	89.07	89.56	90.28	88.86	88.90	88.16	90.41	88.53
156	372	16	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	83.43	85.49	84.57	84.11	88.36	88.90	85.41	84.37	85.14	85.20	91.59	89.12	85.76	86.73	87.82	88.02	87.14	87.13	88.27	87.58	88.95	89.00	89.24	90.10	90.54
159	372	16	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	86.45	84.66	84.54	83.12	86.23	84.71	86.29	86.77	87.14	88.42	87.90	90.15	88.27	85.89	87.39	87.71	87.08	87.84	86.64	92.44	89.17	90.39	86.94	90.74	89.45
208	372	16	0.1	16	0.00100	36	0.000000e+00	0.000000e+00	86.67	87.98	63.59	63.86	62.85	59.20	57.46	61.90	62.14	64.34	58.27	58.80	89.54	60.44	61.16	58.54	61.73	62.59	90.24	59.30	60.42	61.10	59.31	62.18	60.49
211	372	16	0.1	16	0.00100	36	0.000000e+00	1.000000e-08	88.48	61.61	56.27	64.37	43.72	59.39	88.08	59.30	56.11	62.21	58.03	58.90	43.72	58.33	58.84	90.43	61.62	59.41	91.53	57.01	61.19	43.72	58.87	58.82	56.81
220	372	16	0.1	16	0.00100	36	1.000000e-08	0.000000e+00	56.24	63.89	87.93	63.28	64.95	56.29	86.28	62.23	85.18	63.78	61.26	58.82	88.84	62.72	63.11	87.04	57.11	88.61	60.27	56.89	67.23	67.30	55.54	62.53	62.89
223	372	16	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	86.65	55.70	87.44	56.59	63.19	90.64	89.31	61.36	59.41	61.83	91.53	91.85	89.03	59.42	61.08	89.60	61.28	89.44	90.51	56.24	59.78	59.63	60.12	59.77	57.16
272	372	16	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	89.65	60.95	61.79	60.62	61.37	90.47	55.62	56.00	84.71	60.56	92.19	93.81	89.38	92.14	59.70	91.96	59.87	90.09	91.67	58.76	58.89	57.83	90.23	58.01	58.87
275	372	16	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	89.75	59.88	60.05	60.38	90.49	90.23	85.22	91.72	59.53	85.09	89.17	58.76	91.18	59.96	59.59	91.54	54.77	91.43	91.44	92.98	60.29	59.70	58.48	59.14	57.48
284	372	16	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	88.12	56.48	90.11	61.64	62.69	90.30	89.46	59.56	91.30	60.93	91.97	92.55	89.70	90.04	61.86	91.19	55.31	91.03	91.43	92.23	59.09	58.29	89.92	58.82	58.82
287	372	16	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	88.39	90.65	88.41	61.00	88.52	91.99	89.35	59.44	91.54	91.49	90.93	58.79	90.42	91.45	92.63	82.72	58.98	90.62	58.93	91.19	56.88	57.78	59.02	60.00	57.57
336	372	16	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	87.64	61.05	87.87	60.19	90.47	91.06	89.66	59.61	90.93	59.59	92.06	89.31	87.80	58.83	59.21	89.18	58.31	91.43	89.09	91.96	58.79	58.31	59.19	59.65	57.70
339	372	16	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	87.41	90.69	86.65	60.40	87.11	92.18	85.83	59.96	90.15	89.16	89.54	58.52	84.51	57.93	60.38	90.66	91.71	90.19	57.70	58.87	58.25	56.58	90.42	56.61	57.64
348	372	16	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	89.66	60.44	88.66	60.41	89.72	93.22	89.91	60.42	87.90	90.94	58.35	93.18	86.83	91.42	89.92	91.42	59.48	92.97	89.16	92.17	55.85	57.59	92.46	54.75	57.24
351	372	16	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	87.64	59.77	87.64	61.82	89.67	89.14	88.40	89.16	89.93	89.90	91.68	89.03	91.65	87.55	43.72	86.68	90.68	90.42	57.16	89.15	88.62	56.83	58.83	59.35	60.03
400	372	16	0.1	16	0.00100	48	0.000000e+00	0.000000e+00	88.53	62.34	56.40	63.43	63.08	56.76	63.24	62.62	62.40	61.88	59.08	59.72	57.32	62.63	43.72	89.15	62.85	61.47	60.94	60.47	61.38	62.71	61.73	62.73	64.98
403	372	16	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	57.01	60.65	55.84	56.06	58.59	57.09	57.50	61.17	56.04	87.87	90.59	56.27	88.86	59.33	60.32	91.02	59.43	59.68	56.63	56.62	62.23	58.94	59.70	43.72	60.31
412	372	16	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	43.72	87.83	62.18	63.60	63.48	55.91	87.76	61.25	56.83	62.98	89.82	59.61	89.04	61.85	60.79	62.26	63.26	62.29	91.35	58.71	61.80	58.17	63.21	58.59	62.80
415	372	16	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	54.68	54.45	57.68	57.89	57.44	89.79	90.47	59.15	88.09	56.75	53.85	55.47	91.73	56.81	59.04	55.10	89.59	91.21	91.44	53.96	54.75	57.66	51.45	58.45	56.97
464	372	16	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	88.70	60.70	55.52	43.72	60.60	55.78	87.67	59.31	90.25	63.19	57.22	91.96	90.42	91.72	59.67	92.09	60.86	59.37	58.80	58.71	57.69	59.94	58.86	58.11	59.08
467	372	16	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	87.78	62.34	88.81	61.42	61.39	90.89	90.45	61.07	60.33	90.44	91.16	59.53	91.17	60.58	60.47	91.68	60.89	90.72	92.34	86.61	58.47	61.03	53.15	61.66	60.90
476	372	16	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	87.87	62.80	89.32	88.85	89.43	56.05	88.28	61.77	55.96	90.74	88.21	59.78	90.91	90.34	60.50	91.20	59.98	88.90	90.94	58.57	57.60	60.88	87.58	58.89	58.43
479	372	16	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	87.95	61.87	89.97	61.10	88.97	91.42	91.37	59.53	56.52	90.71	92.44	57.58	90.43	58.92	89.22	91.93	91.73	83.67	92.46	90.16	88.59	55.64	89.38	57.92	56.12
528	372	16	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	90.17	90.92	59.26	90.49	88.40	92.18	89.90	90.12	90.70	89.42	92.47	56.30	87.56	60.02	58.48	90.37	59.01	92.74	57.08	58.63	58.60	56.73	56.76	57.91	56.86
531	372	16	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	88.02	88.67	88.41	90.09	89.68	89.27	90.24	59.27	88.01	89.41	89.78	58.16	90.36	90.97	60.61	90.91	59.14	90.90	92.44	90.36	89.38	57.78	58.13	57.38	58.39
540	372	16	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	87.16	61.01	88.87	60.54	88.66	56.81	88.89	58.65	87.67	58.21	88.60	56.72	89.51	89.05	60.25	87.41	59.29	90.92	90.28	59.16	54.51	55.90	56.95	57.13	56.86
543	372	16	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	87.90	60.18	61.16	87.78	88.67	90.23	87.34	57.89	59.40	61.42	57.91	92.93	90.65	57.54	59.93	91.68	59.80	91.17	90.15	87.97	58.14	56.79	56.35	58.34	56.38
592	372	16	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	84.08	80.22	85.36	81.60	85.38	84.58	83.15	82.37	83.81	84.73	86.75	85.44	82.44	88.83	84.74	78.76	87.42	81.54	81.29	87.19	83.10	84.63	76.94	82.69	86.07
595	372	16	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	82.84	84.44	83.41	84.70	87.27	80.76	85.20	86.59	82.64	86.92	87.03	85.98	85.08	86.97	87.88	85.51	89.05	87.15	87.76	64.99	86.49	75.68	88.06	88.69	87.54
604	372	16	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	84.55	83.77	86.09	83.06	85.80	83.10	80.14	85.06	84.33	84.92	61.53	85.98	85.33	87.62	88.92	84.67	87.79	86.36	87.53	87.62	86.86	86.20	86.98	87.39	87.30
607	372	16	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	67.25	69.05	87.80	88.19	88.71	85.38	83.67	90.58	90.27	90.32	88.71	64.42	87.92	88.57	84.30	83.58	87.51	85.72	81.95	72.89	90.53	88.11	88.10	89.10	87.69
656	372	16	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	60.04	87.84	85.51	83.37	86.01	86.65	84.90	55.03	84.99	86.32	86.69	86.24	86.32	86.40	88.24	88.15	86.56	87.50	89.52	89.96	88.13	88.68	89.21	90.17	87.88
659	372	16	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	84.48	86.20	85.22	84.23	82.40	85.49	83.94	86.14	84.96	90.00	85.69	89.30	88.31	90.67	86.05	87.08	86.65	85.99	88.84	91.57	88.34	89.37	88.76	90.95	87.33
668	372	16	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	84.83	83.87	85.88	85.94	84.20	86.71	83.54	88.89	86.68	85.46	90.01	86.19	86.43	86.14	88.75	88.74	89.55	86.60	90.65	87.92	89.38	89.22	88.77	90.98	88.30
671	372	16	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	83.26	86.21	86.70	86.64	86.79	88.41	83.54	87.94	85.52	87.25	86.95	88.36	86.00	87.76	87.40	88.76	85.35	87.11	89.34	87.46	87.33	89.84	90.51	88.31	88.02
720	372	16	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	84.04	86.42	83.62	87.03	84.32	86.52	86.53	85.36	88.58	84.96	87.46	88.42	86.99	86.31	87.64	86.45	90.35	88.16	90.52	91.09	94.45	87.99	88.48	88.75	86.83
723	372	16	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	87.04	89.27	86.62	86.82	86.33	91.31	84.03	89.00	88.66	86.85	90.99	87.54	87.29	87.78	89.05	86.66	87.18	88.10	88.82	93.96	91.12	89.24	87.17	55.98	90.02
732	372	16	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	82.65	85.25	84.79	43.72	83.75	87.82	84.32	88.47	84.22	84.95	90.98	87.10	88.02	88.46	85.99	88.04	86.45	86.56	85.51	87.03	88.77	89.08	87.96	89.26	88.93
735	372	16	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	83.55	85.74	84.63	84.88	82.47	88.00	84.12	87.47	86.10	87.44	90.31	87.43	89.65	85.66	88.47	85.50	90.54	84.64	89.04	89.56	90.07	89.50	88.34	90.37	91.10
784	372	16	0.1	16	0.00010	36	0.000000e+00	0.000000e+00	57.80	63.46	63.16	63.27	63.29	60.56	86.29	63.44	57.12	63.12	59.87	55.93	88.33	61.40	63.02	61.16	62.25	61.92	58.82	60.20	64.54	64.07	65.23	62.91	61.72
787	372	16	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	87.93	61.65	62.84	60.88	61.49	90.04	87.75	59.58	90.33	61.62	85.58	59.11	83.15	62.14	62.18	90.16	59.82	62.45	58.77	91.29	61.49	59.55	61.18	61.92	58.89
796	372	16	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	87.42	64.55	56.67	64.57	65.08	89.72	88.11	63.06	88.27	64.64	90.92	58.25	90.87	63.35	61.66	62.47	63.96	89.42	59.31	59.61	65.19	66.58	60.59	62.29	60.11
799	372	16	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	88.60	55.47	59.84	61.14	60.11	91.07	87.74	60.11	58.79	87.55	90.53	55.31	88.65	58.60	57.76	89.94	57.70	56.64	91.91	54.53	55.49	58.48	57.17	59.56	55.73
848	372	16	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	88.80	60.46	88.99	62.31	61.12	92.25	89.45	90.54	90.01	55.67	91.68	59.18	90.55	91.72	60.28	89.32	60.38	90.76	90.93	91.20	56.95	60.00	59.84	58.69	58.10
851	372	16	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	89.05	61.41	61.72	61.05	89.20	92.43	89.23	90.97	90.68	90.06	92.05	58.77	91.68	90.59	58.42	92.73	58.84	89.71	91.67	92.46	58.02	58.82	90.25	57.72	57.60
860	372	16	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	88.90	62.37	87.99	90.64	90.47	90.93	87.63	59.29	89.57	60.25	58.37	58.48	91.92	59.91	60.78	59.76	59.76	60.41	91.02	92.19	58.95	59.97	54.59	59.87	57.05
863	372	16	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	87.15	89.51	87.86	60.59	89.11	89.70	89.77	60.72	91.09	91.72	56.53	93.59	89.17	59.55	60.05	91.21	60.87	91.19	60.25	88.59	58.13	57.64	83.08	60.50	58.19
912	372	16	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	88.66	62.10	87.68	89.15	89.47	92.36	86.60	89.37	90.15	82.22	56.01	59.21	91.43	91.94	59.78	90.66	90.92	91.72	92.17	58.52	55.65	56.58	57.43	59.02	56.83
915	372	16	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	89.67	60.69	89.19	55.25	88.08	57.39	87.82	89.52	86.32	90.52	89.88	57.22	89.93	58.78	58.55	85.10	91.40	90.02	91.30	58.07	58.16	57.85	57.21	58.07	58.25
924	372	16	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	90.20	88.68	63.43	88.68	89.20	90.89	85.79	58.90	89.17	90.71	91.40	56.33	89.36	90.71	60.14	91.19	59.64	89.66	90.66	59.17	56.23	56.00	57.16	56.15	58.95
927	372	16	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	88.46	87.09	89.66	60.96	89.22	91.97	85.83	87.38	90.35	91.21	58.52	60.25	87.04	89.10	60.31	88.72	59.70	88.67	90.39	91.17	58.57	57.66	88.05	57.70	59.64
976	372	16	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	64.19	64.25	62.22	63.34	62.41	89.37	64.64	63.09	58.07	64.32	57.64	59.65	60.04	59.42	62.28	58.46	58.69	64.48	57.33	43.72	65.85	66.05	57.41	63.48	66.01
979	372	16	0.1	16	0.00010	48	0.000000e+00	1.000000e-08	62.60	61.93	87.08	60.63	63.52	90.94	56.10	59.82	56.79	62.36	91.93	55.98	89.19	60.40	61.00	91.38	61.82	59.26	91.14	55.69	60.80	61.43	62.16	59.84	58.27
988	372	16	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	56.63	63.28	62.66	63.29	62.46	89.32	88.45	62.17	61.09	62.96	90.19	59.02	87.63	61.79	64.04	56.78	89.07	91.77	89.87	58.42	61.66	65.03	63.89	60.93	61.96
991	372	16	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	86.34	61.44	61.39	63.30	59.70	91.40	90.39	61.41	62.86	62.05	57.49	91.09	88.18	57.91	61.65	60.25	61.01	56.99	90.28	54.75	60.40	61.12	58.79	60.37	59.72
1040	372	16	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	88.89	61.17	61.18	61.33	62.57	59.03	89.46	60.20	62.91	92.13	91.40	58.54	89.80	91.72	60.77	92.84	61.25	92.10	59.04	56.06	60.51	58.09	56.87	58.34	59.83
1043	372	16	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	88.74	61.30	56.02	60.76	90.23	91.55	90.83	89.82	87.79	90.48	92.69	92.21	92.17	90.13	60.65	90.16	91.23	60.76	91.68	58.62	60.26	59.37	91.31	59.32	59.55
1052	372	16	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	89.22	56.29	89.10	89.55	89.98	92.03	88.29	86.85	90.51	91.39	91.67	91.15	85.73	91.43	56.79	91.73	92.20	91.14	91.67	93.30	57.33	59.70	89.95	59.53	58.52
1055	372	16	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	89.75	88.67	60.58	87.11	60.68	91.80	89.75	43.72	60.86	91.45	59.19	59.37	90.27	59.40	61.51	91.80	61.35	92.22	89.20	59.16	57.07	58.30	53.28	59.32	57.96
1104	372	16	0.1	64	0.00010	48	0.000000e+00	0.000000e+00	88.94	89.41	88.09	60.53	89.18	58.62	90.67	58.39	90.22	90.21	56.03	55.53	90.40	91.95	58.85	86.17	58.86	90.99	58.60	56.45	57.40	58.71	58.16	58.58	58.89
1107	372	16	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	87.39	89.69	88.72	61.07	89.93	92.22	86.84	59.25	90.46	59.42	90.13	58.04	90.35	91.92	88.77	91.27	60.30	90.42	90.62	90.63	57.06	57.47	57.67	59.67	57.11
1116	372	16	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	88.17	61.95	89.02	60.16	90.69	90.38	89.91	60.50	58.85	58.96	89.03	57.12	91.80	58.10	59.08	88.90	59.93	56.06	89.62	58.76	57.73	56.81	57.31	56.93	56.57
1119	372	16	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	87.26	61.63	89.40	90.71	90.93	90.38	88.92	60.44	89.36	88.89	89.74	91.42	91.67	90.17	59.04	86.90	92.17	88.38	90.63	57.64	56.45	57.74	57.84	56.86	87.26
1168	372	16	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	43.72	86.56	82.78	81.68	68.74	86.34	61.11	87.07	87.71	84.40	82.38	82.05	81.26	63.91	82.83	69.15	88.41	86.18	61.20	85.54	87.29	87.00	79.27	85.87	77.39
1171	372	16	0.1	16	0.00001	24	0.000000e+00	1.000000e-08	82.90	82.44	84.01	79.45	83.91	88.70	84.51	83.88	83.37	88.11	85.76	87.61	86.01	83.45	83.75	86.35	85.57	85.18	90.24	86.68	62.99	61.89	85.35	85.05	87.22
1180	372	16	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	83.16	63.55	83.52	87.32	63.13	88.14	84.58	84.08	85.55	83.82	87.90	90.75	81.58	59.82	62.04	86.25	85.98	84.67	88.69	88.70	88.30	87.71	87.13	82.29	81.74
1183	372	16	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	68.40	82.34	82.16	85.50	83.07	83.79	82.39	85.47	83.02	85.78	85.08	86.11	86.38	84.11	87.14	86.09	87.26	85.08	84.01	87.60	86.39	85.59	87.16	83.17	89.30
1232	372	16	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	83.53	88.44	84.88	85.08	85.08	89.43	84.85	90.33	84.04	87.40	86.57	87.65	85.39	88.88	88.49	87.10	87.04	87.07	88.05	90.21	89.57	89.90	86.93	88.73	88.68
1235	372	16	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	88.08	88.34	83.96	87.54	88.65	88.35	85.47	85.72	84.54	88.33	87.71	86.85	82.69	87.69	88.06	86.95	87.56	88.23	87.94	88.60	88.42	89.57	88.92	88.46	89.61
1244	372	16	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	86.99	56.21	89.61	87.22	85.27	86.52	84.44	86.52	84.84	87.05	89.35	88.93	86.18	86.93	90.46	88.28	90.32	87.66	87.77	92.33	87.06	87.15	89.89	89.03	87.86
1247	372	16	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	84.47	83.01	85.60	85.18	85.28	87.60	85.33	86.50	87.33	87.77	89.77	90.62	84.97	87.59	85.17	86.55	88.49	85.67	90.47	91.00	86.41	90.04	88.63	89.31	91.02
1296	372	16	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	83.64	86.70	86.16	86.01	83.74	84.24	83.45	83.75	86.29	84.99	90.09	87.13	85.99	87.26	87.12	84.36	88.75	85.88	91.90	88.39	89.80	87.01	89.01	88.75	86.52
1299	372	16	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	83.54	87.87	85.84	87.11	84.48	89.90	87.09	83.97	85.00	87.07	87.90	88.09	85.57	87.65	85.51	85.62	86.40	85.91	88.11	87.06	87.29	87.76	87.12	92.64	87.12
1308	372	16	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	84.38	84.40	86.42	87.16	83.94	89.47	85.43	86.50	87.49	88.84	92.23	85.77	87.70	84.63	84.99	88.64	88.38	85.77	88.99	87.56	87.51	87.63	86.90	94.54	89.58
1311	372	16	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	84.86	88.25	85.90	83.87	84.58	85.85	85.17	88.22	85.54	86.28	91.55	88.28	88.10	85.41	86.97	87.23	86.29	87.53	91.86	87.80	89.01	89.03	87.46	88.34	90.09
1360	372	16	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	65.58	58.26	63.46	62.16	66.03	61.05	57.26	62.22	63.11	56.73	58.81	59.38	56.50	60.51	61.80	89.82	64.48	61.51	59.32	61.35	61.24	64.88	60.92	62.68	60.62
1363	372	16	0.1	16	0.00001	36	0.000000e+00	1.000000e-08	86.64	63.64	65.93	62.95	64.53	61.69	56.73	64.68	63.77	66.23	91.91	59.93	88.64	57.57	61.05	89.95	61.53	62.59	59.80	60.25	61.85	62.90	61.99	62.19	62.29
1372	372	16	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	57.62	65.66	64.73	67.80	65.03	62.65	87.39	65.15	87.94	64.51	89.76	62.30	65.86	64.76	64.38	64.11	64.38	89.94	88.68	62.78	63.23	62.34	63.13	62.31	63.40
1375	372	16	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	56.73	61.81	63.26	63.38	61.45	58.20	86.69	60.27	60.53	62.54	56.65	59.07	89.38	89.17	59.90	60.82	61.69	63.73	58.32	55.22	61.92	65.23	59.44	59.26	58.35
1424	372	16	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	89.43	59.37	60.86	90.31	60.29	91.97	89.47	60.36	89.05	59.76	92.47	57.77	92.39	91.04	60.33	92.93	83.96	90.55	91.55	90.95	91.35	57.92	57.50	58.23	59.09
1427	372	16	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	88.66	62.08	86.90	61.21	61.26	90.62	88.46	60.64	90.54	61.59	91.46	91.96	90.92	91.47	62.54	91.10	60.73	91.16	92.97	58.62	59.00	59.07	56.94	59.82	59.26
1436	372	16	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	88.43	61.33	88.44	88.77	88.55	89.92	87.41	61.70	62.12	91.44	91.11	90.45	87.65	91.46	60.53	89.66	61.29	91.17	58.10	91.29	58.26	57.54	61.57	61.55	59.67
1439	372	16	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	88.21	89.50	88.75	59.01	91.13	91.66	91.21	60.61	90.19	58.20	89.71	92.18	90.96	59.26	60.79	91.55	90.78	91.69	89.00	92.93	60.76	57.98	59.35	57.82	57.90
1488	372	16	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	89.14	61.27	88.17	89.93	61.05	87.32	90.16	60.27	91.91	88.92	89.61	81.34	89.91	60.86	58.94	85.71	57.72	92.71	89.90	57.22	57.08	56.61	89.86	57.30	55.88
1491	372	16	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	84.26	61.17	59.80	60.26	88.17	91.72	89.41	90.90	91.95	89.42	88.79	89.80	90.45	90.92	59.30	91.43	58.08	90.67	91.19	90.92	58.06	54.86	58.52	58.95	59.23
1500	372	16	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	90.54	89.65	88.40	60.17	88.71	91.64	89.95	59.78	87.36	90.44	90.13	88.29	87.89	59.52	59.36	92.22	58.65	90.68	56.51	59.00	55.90	59.04	57.28	56.17	59.28
1503	372	16	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	89.43	89.17	88.15	89.16	89.32	90.86	85.88	58.54	90.37	59.20	56.75	59.20	88.27	61.32	59.48	92.18	59.51	90.31	93.47	92.35	89.52	58.12	57.56	56.73	58.39
1552	372	16	0.1	16	0.00001	48	0.000000e+00	0.000000e+00	86.07	61.52	63.32	62.94	61.64	57.08	88.35	60.37	63.45	57.01	55.30	55.59	89.58	57.50	88.09	49.75	59.44	61.29	58.14	57.81	58.31	62.57	61.24	53.44	59.37
1555	372	16	0.1	16	0.00001	48	0.000000e+00	1.000000e-08	87.84	64.67	87.41	62.95	61.72	60.18	86.54	62.31	88.40	62.24	91.32	90.85	90.50	59.14	60.14	57.25	88.47	89.08	92.25	58.25	59.67	59.51	53.32	59.97	61.04
1564	372	16	0.1	16	0.00001	48	1.000000e-08	0.000000e+00	56.36	57.80	86.24	86.09	60.49	91.16	88.36	53.19	57.73	89.32	90.32	90.66	89.00	57.70	55.04	91.16	58.54	56.06	92.20	93.10	52.23	57.19	57.32	57.00	56.02
1567	372	16	0.1	16	0.00001	48	1.000000e-08	1.000000e-08	87.90	60.15	62.06	88.05	61.51	88.26	87.44	60.39	62.48	56.69	90.36	54.66	91.42	60.59	62.04	58.14	59.96	60.43	58.37	91.61	60.21	57.99	59.33	56.67	59.48

Test Data mAP Results Preview:

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_zero	mAP_test_zero_2	mAP_test_zero_3	mAP_test_zero_4	mAP_test_zero_5	mAP_test_zero_6	mAP_test_zero_7	mAP_test_zero_8	mAP_test_zero_9	mAP_test_zero_10	mAP_test_zero_11	mAP_test_zero_12	mAP_test_zero_13	mAP_test_zero_14	mAP_test_zero_15	mAP_test_zero_16	mAP_test_zero_17	mAP_test_zero_18	mAP_test_zero_19	mAP_test_zero_20	mAP_test_zero_21	mAP_test_zero_22	mAP_test_zero_23	mAP_test_zero_25	mAP_test_zero_26
16	372	16	0.1	16	0.00100	24	0.000000e+00	0.000000e+00	89.17	53.79	91.55	93.05	90.65	92.51	90.87	88.72	92.76	54.02	90.24	80.28	90.89	91.06	92.20	89.63	90.72	88.56	83.79	84.41	60.96	86.42	62.30	88.38	93.90
19	372	16	0.1	16	0.00100	24	0.000000e+00	1.000000e-08	94.06	50.70	90.25	80.07	84.43	91.49	93.12	64.15	90.43	92.96	95.24	91.28	92.38	91.85	66.91	92.09	60.25	94.16	92.23	89.11	91.47	57.08	61.26	91.58	89.16
28	372	16	0.1	16	0.00100	24	1.000000e-08	0.000000e+00	91.30	90.74	92.72	61.08	93.34	92.90	53.94	92.45	89.37	82.76	94.72	89.36	91.72	90.53	81.94	87.87	89.35	91.35	90.79	88.35	93.68	92.18	53.79	92.31	91.59
31	372	16	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	93.27	85.60	90.88	52.35	86.17	91.73	93.31	91.27	92.84	92.68	89.67	91.14	91.28	93.30	92.26	92.65	48.12	93.40	90.74	88.92	89.07	91.17	92.01	91.71	93.39
80	372	16	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	89.84	95.59	89.30	92.87	93.64	91.45	93.72	94.63	96.05	95.10	93.32	85.38	93.42	48.12	58.32	78.79	94.49	91.95	93.05	92.38	60.28	92.39	93.12	92.61	91.61
83	372	16	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	94.39	90.90	90.83	91.65	95.27	92.45	94.89	93.57	94.36	93.48	92.70	92.33	92.09	94.71	93.37	95.36	90.57	91.30	92.74	92.52	58.52	92.53	91.34	92.05	93.06
92	372	16	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	93.84	90.64	93.37	91.47	93.39	92.08	94.27	93.29	94.14	91.98	93.36	92.09	91.38	94.82	87.58	93.06	93.73	92.79	92.79	94.12	92.66	95.30	91.86	52.60	59.48
95	372	16	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	95.19	89.90	91.66	93.81	92.61	91.11	93.92	92.79	92.28	92.95	91.71	93.93	94.13	91.67	94.56	93.28	96.33	93.17	92.52	92.19	93.98	93.16	93.88	92.57	68.90
144	372	16	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	94.08	92.56	95.00	91.98	95.50	89.95	92.75	93.10	94.56	93.18	92.63	92.37	94.71	94.65	88.54	92.71	48.12	93.22	89.43	91.09	91.74	92.48	92.68	56.57	92.61
147	372	16	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	94.33	96.36	91.19	93.37	95.37	92.15	95.07	93.06	93.12	93.08	93.84	92.27	91.78	86.90	48.12	95.65	48.12	91.03	92.12	92.25	93.78	93.49	92.86	91.47	92.20
156	372	16	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	91.04	94.11	90.14	92.09	94.27	93.11	93.46	91.67	95.06	94.48	92.71	92.29	94.73	92.03	94.63	94.35	93.27	94.86	92.34	92.36	93.98	94.02	94.16	92.60	90.30
159	372	16	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	92.42	88.17	93.81	88.32	93.18	93.06	92.77	93.30	94.09	93.37	91.78	92.68	91.98	95.89	94.92	93.97	92.23	93.10	93.33	91.20	93.22	92.72	92.72	90.34	93.27
208	372	16	0.1	16	0.00100	36	0.000000e+00	0.000000e+00	58.41	59.22	60.50	61.74	62.06	59.74	49.38	60.53	60.71	60.21	59.46	60.54	59.11	59.87	61.42	49.58	59.66	59.76	47.91	59.97	62.93	61.82	63.18	61.35	61.92
211	372	16	0.1	16	0.00100	36	0.000000e+00	1.000000e-08	58.00	60.91	50.36	61.08	48.12	60.33	59.07	60.81	48.91	59.60	60.76	60.69	48.12	59.02	59.90	57.97	60.24	59.32	58.52	60.04	60.69	48.12	61.71	62.89	61.72
220	372	16	0.1	16	0.00100	36	1.000000e-08	0.000000e+00	50.38	61.18	59.53	61.54	61.92	49.64	58.89	62.02	58.50	61.30	61.65	62.34	58.87	61.04	60.28	59.91	48.56	59.30	61.10	50.39	60.60	60.17	51.60	62.06	61.79
223	372	16	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	57.54	49.78	59.67	49.51	61.29	58.27	59.22	61.44	59.17	58.93	58.94	59.01	58.12	58.30	60.13	57.29	60.11	57.24	57.32	60.46	61.06	60.99	61.99	62.59	62.45
272	372	16	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	59.13	61.71	61.17	61.65	62.19	59.84	49.92	50.31	93.02	61.47	60.00	60.01	60.50	59.16	61.99	59.22	60.74	56.91	58.54	61.51	61.90	61.47	59.72	62.73	62.63
275	372	16	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	60.15	61.26	61.70	61.53	58.55	49.01	91.90	59.13	61.76	90.60	50.30	61.93	59.76	60.81	61.86	59.53	49.50	59.81	60.27	59.61	61.78	61.22	62.20	62.65	62.50
284	372	16	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	59.56	50.81	59.58	61.70	61.99	59.85	59.30	62.70	59.29	61.99	59.71	59.47	59.91	58.97	60.83	59.24	48.63	58.33	59.95	59.65	61.43	62.59	60.43	62.79	61.59
287	372	16	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	58.39	59.87	59.27	62.34	58.04	59.76	59.26	60.81	59.23	59.02	60.27	61.99	59.34	58.97	59.24	88.45	61.26	59.33	62.19	59.67	62.48	61.77	61.64	62.06	62.83
336	372	16	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	60.37	61.99	60.27	61.93	59.31	59.63	59.95	62.88	59.57	61.76	60.22	60.94	61.31	61.03	61.50	51.78	61.61	59.56	61.13	59.16	61.45	62.34	62.26	62.32	62.44
339	372	16	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	59.69	59.80	59.77	62.57	60.67	59.88	50.94	62.16	60.37	59.71	61.20	62.06	91.19	62.41	61.99	59.90	59.43	59.79	62.57	62.64	62.60	62.55	59.72	62.66	62.64
348	372	16	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	59.86	63.40	59.77	61.99	59.31	59.76	60.29	62.26	90.90	59.79	62.64	60.11	50.28	59.85	59.95	60.11	62.09	59.23	51.82	59.99	62.55	62.34	59.05	50.08	62.30
351	372	16	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	59.85	62.64	60.19	62.06	59.95	60.77	59.53	59.95	59.83	60.09	59.90	61.20	59.62	90.22	48.12	51.38	59.66	59.65	62.64	60.67	60.26	63.04	62.79	62.77	61.85
400	372	16	0.1	16	0.00100	48	0.000000e+00	0.000000e+00	58.23	61.42	50.98	61.27	60.37	50.98	60.97	60.58	61.09	59.97	59.64	61.03	48.74	60.63	48.12	59.02	61.78	61.13	61.31	60.53	62.71	62.19	62.31	61.80	61.20
403	372	16	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	50.32	60.97	50.23	49.71	60.79	49.58	50.76	60.23	50.46	57.10	58.12	60.42	57.96	60.90	60.63	58.96	60.82	59.09	50.55	49.75	61.34	61.35	59.80	48.12	61.06
412	372	16	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	48.12	58.35	60.46	61.51	60.20	50.50	57.90	61.76	48.44	61.43	60.00	61.09	59.24	61.41	60.82	60.51	61.22	61.61	58.66	61.17	62.75	63.77	62.60	62.36	62.33
415	372	16	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	49.89	50.35	59.81	60.23	59.83	58.92	57.83	60.34	56.28	59.05	51.46	49.82	57.73	59.40	59.78	50.38	57.17	57.42	56.98	51.21	60.14	60.89	52.23	61.27	61.76
464	372	16	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	59.76	62.32	50.50	48.12	62.09	49.74	58.89	61.53	59.13	61.42	62.64	59.62	59.38	59.75	60.55	59.72	61.50	61.76	61.65	61.29	62.55	61.17	62.76	62.88	61.90
467	372	16	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	59.49	62.32	59.59	61.22	62.00	60.37	59.45	61.42	62.00	59.13	60.27	61.99	59.52	60.95	62.00	59.87	60.67	58.39	60.01	84.80	61.85	61.88	53.93	61.82	61.95
476	372	16	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	59.57	61.99	58.85	59.80	59.61	49.46	57.71	62.06	50.01	59.36	61.70	61.75	59.30	58.53	61.18	58.92	61.08	59.76	60.48	61.93	61.28	62.09	60.74	62.72	61.88
479	372	16	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	59.23	60.81	59.32	62.32	59.43	59.85	59.22	61.54	50.51	58.64	59.95	61.59	59.54	60.61	59.34	59.42	59.30	89.32	59.52	58.53	60.79	63.09	60.23	63.04	63.06
528	372	16	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	59.56	59.80	62.32	59.50	60.12	60.01	60.08	60.50	59.69	59.86	59.71	62.84	60.73	61.53	61.70	60.53	61.56	59.58	62.97	62.06	62.66	63.04	62.57	62.03	62.53
531	372	16	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	59.96	60.17	59.47	59.36	59.57	61.30	60.00	61.52	60.83	60.03	61.07	62.41	60.29	60.08	61.48	60.33	61.01	60.27	59.80	88.43	60.26	63.02	62.45	62.36	61.87
540	372	16	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	60.84	61.99	60.12	61.93	59.77	62.90	59.95	63.26	51.66	61.70	61.13	62.26	60.94	61.20	61.99	61.20	61.23	60.03	51.00	62.32	50.26	62.82	62.87	63.16	62.84
543	372	16	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	59.91	62.39	62.16	60.38	60.26	60.87	60.34	62.41	62.41	62.09	62.32	59.95	60.09	62.75	61.11	59.33	61.76	59.71	60.32	51.11	62.32	62.66	62.75	62.52	62.97
592	372	16	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	90.89	87.19	91.88	92.19	92.40	90.84	90.40	91.41	91.95	55.90	93.62	89.64	90.60	57.93	54.74	90.15	85.03	89.90	83.56	89.80	89.05	58.56	65.99	85.74	57.15
595	372	16	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	91.26	93.07	93.93	91.50	93.65	89.62	80.45	85.15	93.88	95.17	89.31	92.46	93.00	92.11	76.38	89.63	91.97	91.53	91.45	92.45	91.90	75.02	90.43	90.16	92.62
604	372	16	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	92.92	95.24	92.46	92.29	90.71	85.02	86.14	92.72	92.47	90.30	89.77	92.31	92.05	91.62	59.66	93.62	93.82	53.12	92.74	92.25	92.28	88.82	92.45	92.28	91.07
607	372	16	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	92.03	89.70	93.79	87.17	92.63	92.17	94.31	90.61	90.18	91.18	92.74	88.70	89.96	90.60	59.16	89.14	59.19	92.65	84.24	85.92	90.98	90.51	90.19	91.88	59.20
656	372	16	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	92.69	64.33	95.27	83.76	93.84	89.18	94.57	91.28	91.61	91.73	92.08	88.68	90.58	92.47	88.55	92.43	87.03	92.24	91.34	92.05	94.50	89.64	91.25	92.14	93.63
659	372	16	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	92.94	93.02	94.02	91.73	93.67	91.16	93.56	92.34	93.53	94.10	91.68	93.23	90.68	93.24	85.87	93.77	91.67	94.26	91.34	93.87	94.04	92.40	91.06	59.76	92.98
668	372	16	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	90.09	89.83	94.95	92.58	94.22	92.98	93.77	59.19	89.68	94.24	93.43	90.63	95.19	92.18	93.14	93.84	91.38	91.87	92.23	94.45	93.09	92.76	91.65	60.71	93.31
671	372	16	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	94.85	93.50	94.33	93.60	88.74	92.37	95.52	91.14	93.30	93.32	93.02	91.58	94.45	94.75	93.96	90.74	89.82	93.46	92.81	89.46	91.90	58.98	93.01	92.55	91.92
720	372	16	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	92.67	94.38	93.64	94.11	94.49	93.21	91.39	89.39	93.59	93.39	94.72	93.42	94.50	87.86	93.49	91.52	91.68	93.76	91.13	92.58	61.03	93.21	94.43	93.94	89.78
723	372	16	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	95.18	92.56	88.21	90.99	91.86	92.64	94.47	93.97	92.58	92.19	92.14	91.85	92.77	94.48	51.17	94.34	94.34	92.46	92.37	93.06	61.74	92.58	91.18	94.59	92.67
732	372	16	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	92.88	91.41	90.16	48.12	93.75	93.14	93.74	91.06	93.60	92.77	93.67	90.63	92.24	90.56	95.45	87.41	91.94	92.89	90.92	91.74	91.13	91.46	93.77	92.72	91.90
735	372	16	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	95.10	93.45	95.77	91.04	89.61	92.23	93.37	92.65	92.79	93.95	89.07	93.26	90.39	92.84	91.41	93.56	90.51	90.42	92.16	92.91	92.24	92.53	61.10	92.86	70.81
784	372	16	0.1	16	0.00010	36	0.000000e+00	0.000000e+00	49.59	60.88	60.23	60.24	61.56	61.62	57.51	60.71	49.69	61.03	60.62	50.37	57.60	60.74	60.07	62.32	60.48	61.29	59.91	60.67	62.05	61.05	61.18	61.68	62.21
787	372	16	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	57.42	61.60	60.77	60.66	60.85	59.81	58.00	59.61	58.49	60.71	51.39	62.03	90.11	59.51	61.20	59.22	60.69	59.64	61.31	58.30	62.28	62.75	62.08	61.95	61.91
796	372	16	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	58.55	61.60	49.53	61.75	60.88	58.07	58.93	61.07	58.70	60.20	59.43	61.65	58.54	60.86	60.16	60.98	60.32	57.33	60.47	61.29	61.48	61.68	61.63	61.90	61.46
799	372	16	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	58.01	48.94	59.93	59.32	59.95	57.79	56.60	59.78	58.79	56.55	58.80	49.92	57.28	59.53	59.07	56.76	59.98	48.74	57.70	50.00	59.72	59.96	63.10	61.11	60.30
848	372	16	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	59.79	60.97	58.64	61.99	61.87	59.63	58.63	59.17	58.51	49.68	59.66	62.32	59.43	59.56	61.65	59.57	60.29	58.74	60.03	59.07	62.48	62.14	62.34	62.99	61.94
851	372	16	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	59.99	61.99	61.65	61.93	59.41	59.71	59.19	59.66	58.19	58.54	60.06	62.70	59.47	59.61	61.48	59.04	62.56	59.64	60.15	58.56	60.63	61.83	59.88	62.58	62.11
860	372	16	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	58.97	61.94	59.22	59.81	59.25	59.80	51.79	63.04	59.57	61.83	61.99	62.06	59.62	62.55	61.09	61.07	60.99	61.70	60.39	59.91	61.90	62.02	52.12	62.44	62.66
863	372	16	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	59.70	59.15	59.57	62.39	60.27	60.02	59.38	62.32	59.42	59.56	62.97	59.32	59.97	60.72	61.93	59.58	61.70	59.52	61.93	50.96	62.34	62.57	87.88	62.15	62.13
912	372	16	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	59.91	62.06	59.85	60.22	59.62	59.02	93.12	60.03	60.08	90.77	50.51	61.71	59.62	59.21	62.06	60.37	59.90	59.62	60.27	61.99	62.25	62.69	62.39	61.78	62.75
915	372	16	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	59.88	62.32	59.57	50.86	59.43	62.64	92.75	59.96	51.47	59.61	60.67	62.39	60.09	61.99	61.43	92.17	59.81	59.57	60.01	61.99	62.42	62.92	62.75	62.66	61.62
924	372	16	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	59.45	59.70	61.74	59.72	59.21	60.33	61.20	62.41	59.81	59.50	51.47	62.72	60.34	58.92	61.87	59.85	61.61	59.98	60.22	62.32	61.55	62.32	62.79	62.50	61.81
927	372	16	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	59.42	52.56	59.93	62.39	59.43	59.33	60.53	50.74	60.65	59.57	62.97	61.93	51.36	60.72	61.76	50.40	61.66	60.86	60.83	59.53	62.36	62.57	60.44	63.09	62.44
976	372	16	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	59.23	61.33	61.34	61.19	60.65	57.11	61.44	61.07	49.14	60.88	59.40	61.60	48.37	59.62	60.61	49.16	63.79	60.59	50.39	48.12	60.94	61.18	51.39	61.94	61.50
979	372	16	0.1	16	0.00010	48	0.000000e+00	1.000000e-08	60.48	60.78	57.58	61.44	61.76	58.02	50.37	60.34	49.20	61.05	59.15	49.82	58.51	60.91	60.65	57.99	60.52	59.01	58.87	49.97	62.15	61.82	62.02	61.69	62.16
988	372	16	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	50.20	61.29	61.50	61.78	60.07	58.67	58.29	61.98	60.87	61.33	60.30	61.12	59.32	59.49	60.30	50.16	59.31	58.67	58.28	61.06	61.87	61.39	61.54	62.96	62.35
991	372	16	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	57.86	60.77	60.52	61.33	59.70	48.01	58.47	60.85	60.41	58.75	60.00	56.63	57.73	58.78	60.58	60.01	59.03	50.57	56.11	50.25	61.76	62.11	61.63	61.96	61.10
1040	372	16	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	59.29	61.65	62.00	61.66	61.36	61.61	59.11	61.36	61.64	59.57	60.38	61.91	59.00	58.98	61.93	58.56	61.64	58.79	62.32	61.38	61.74	62.41	62.10	62.32	61.43
1043	372	16	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	59.62	61.93	50.73	61.93	59.34	59.86	59.41	59.86	52.41	59.37	60.01	59.80	59.67	58.37	61.11	60.27	58.97	61.45	59.96	61.99	61.38	61.92	59.43	62.46	61.81
1052	372	16	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	59.47	49.98	58.85	59.81	60.23	59.78	57.95	51.19	59.11	59.57	60.38	60.49	52.74	60.08	49.32	58.63	58.40	59.71	59.90	59.87	62.55	60.78	59.69	61.83	62.45
1055	372	16	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	59.78	59.83	61.67	60.72	61.59	59.83	59.00	48.12	61.93	59.83	62.64	62.32	59.92	60.66	61.74	59.28	61.60	58.92	61.13	61.87	62.67	62.55	52.64	62.24	62.26
1104	372	16	0.1	64	0.00010	48	0.000000e+00	0.000000e+00	59.29	60.27	59.46	61.70	59.52	62.64	59.79	62.34	59.52	59.45	62.97	63.04	60.09	59.62	60.89	50.93	61.77	59.10	62.71	62.32	62.90	61.76	62.26	61.93	62.12
1107	372	16	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	60.43	59.65	59.41	61.99	59.81	59.45	60.40	62.90	59.46	61.77	60.73	62.32	60.60	59.86	52.81	59.76	61.64	59.61	60.77	60.77	62.48	62.39	62.60	62.32	62.97
1116	372	16	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	59.94	62.26	60.58	61.93	59.86	60.61	59.87	62.05	62.64	62.20	61.43	62.64	59.62	62.64	61.01	60.38	62.19	49.50	60.78	61.93	62.27	62.43	62.27	62.73	62.97
1119	372	16	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	60.32	61.01	59.62	59.48	59.80	60.77	59.48	62.26	60.77	60.89	49.99	60.00	59.57	60.03	61.45	90.79	59.71	60.81	60.77	61.94	62.91	62.03	62.71	62.68	51.89
1168	372	16	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	48.12	90.43	92.68	91.27	93.12	88.49	93.09	90.24	91.91	93.33	83.18	84.26	93.35	90.98	55.30	89.96	93.56	91.11	91.15	92.44	88.83	90.90	88.93	92.22	89.65
1171	372	16	0.1	16	0.00001	24	0.000000e+00	1.000000e-08	93.36	90.00	94.23	89.30	95.29	57.43	92.12	59.09	94.51	93.18	92.68	90.24	92.49	75.69	76.10	93.72	58.12	93.65	76.69	89.83	90.29	90.65	93.71	87.09	92.58
1180	372	16	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	91.77	91.95	93.32	93.64	89.65	92.47	94.61	59.41	93.56	93.96	94.22	52.88	90.72	93.75	89.49	92.58	91.26	58.94	90.03	88.90	92.46	92.73	58.95	90.98	55.73
1183	372	16	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	90.13	91.74	93.70	90.35	93.46	88.42	89.63	93.10	92.42	92.38	89.48	88.16	94.35	92.66	91.64	90.89	58.20	89.07	91.24	87.34	92.33	90.88	91.66	53.81	59.89
1232	372	16	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	91.98	90.81	94.19	90.04	89.78	91.23	93.26	91.17	93.62	89.31	93.19	87.83	94.32	92.77	70.81	94.43	90.43	90.22	90.41	91.43	82.66	92.47	92.91	91.48	90.61
1235	372	16	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	89.49	86.62	93.62	67.86	93.02	92.15	93.44	90.97	90.61	94.83	92.54	89.83	93.51	91.92	89.66	93.63	96.14	94.86	94.15	91.40	93.39	93.46	93.06	59.77	91.23
1244	372	16	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	91.54	92.46	93.30	91.73	95.96	93.78	93.88	93.63	89.45	93.43	91.94	90.88	90.20	91.56	90.04	94.46	92.45	91.51	93.73	91.80	92.01	86.17	92.01	91.54	92.13
1247	372	16	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	94.20	95.16	93.76	92.47	94.85	94.95	91.46	93.41	94.73	94.36	93.38	91.38	90.08	89.94	93.46	93.53	95.17	92.90	93.26	93.37	91.25	92.30	93.47	91.38	88.62
1296	372	16	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	94.60	93.79	93.58	92.35	92.14	93.49	93.55	91.89	77.34	90.85	91.62	93.45	93.81	94.03	94.46	85.01	94.67	94.63	93.33	91.36	94.09	86.81	93.25	91.70	85.41
1299	372	16	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	94.48	91.95	94.10	95.68	91.33	93.16	95.01	96.11	94.37	91.72	93.19	92.23	93.95	94.74	94.36	95.88	91.65	92.78	92.42	94.19	94.52	93.49	92.28	60.84	93.55
1308	372	16	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	94.05	91.36	94.72	92.68	90.38	93.68	93.75	92.71	92.64	94.20	93.13	94.55	94.07	93.98	91.67	92.87	92.13	94.84	94.51	92.39	92.34	91.87	89.13	62.26	87.81
1311	372	16	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	93.36	92.41	91.80	89.90	90.39	92.37	94.18	94.33	93.04	93.96	91.64	93.59	92.92	95.46	92.38	92.40	90.95	92.07	92.96	93.30	91.36	95.07	88.88	93.09	92.51
1360	372	16	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	61.20	48.90	60.68	60.34	60.80	61.34	49.90	60.84	60.74	50.70	60.59	59.25	49.51	59.62	61.57	59.39	61.79	59.63	59.49	61.96	62.40	61.23	62.73	61.43	62.38
1363	372	16	0.1	16	0.00001	36	0.000000e+00	1.000000e-08	59.97	60.74	61.04	61.79	61.11	61.37	50.49	61.29	60.47	61.46	58.84	62.32	59.16	49.62	61.02	58.19	60.57	58.55	61.67	61.99	62.38	62.24	61.84	62.13	62.17
1372	372	16	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	48.74	61.59	61.38	61.09	61.73	61.47	59.39	60.01	58.37	59.75	60.49	61.65	61.10	61.00	60.76	60.65	61.59	58.67	48.75	62.06	61.81	62.43	61.83	62.01	62.23
1375	372	16	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	50.26	59.99	60.61	61.01	60.13	59.87	56.90	59.66	59.59	59.80	49.42	61.37	58.25	57.37	60.12	61.70	60.83	60.36	60.56	50.72	62.05	61.43	60.54	62.63	61.77
1424	372	16	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	58.84	60.78	61.47	59.25	61.70	59.43	58.95	61.75	58.57	61.66	59.62	61.93	59.53	59.87	61.41	59.27	90.45	58.23	58.15	58.63	59.16	63.20	62.67	62.50	61.37
1427	372	16	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	59.52	61.87	59.93	61.71	61.94	60.61	59.14	62.14	58.97	61.99	59.25	59.52	59.67	59.52	60.63	59.87	62.28	59.01	59.48	61.90	62.32	62.55	50.39	61.94	62.03
1436	372	16	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	59.18	62.08	59.27	59.20	59.79	59.92	59.49	61.71	62.06	59.41	59.90	59.58	59.99	58.74	61.76	59.99	61.55	59.33	62.64	59.88	62.45	62.55	61.35	62.46	61.76
1439	372	16	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	58.99	59.72	59.33	61.59	59.20	59.78	59.23	62.24	58.32	61.98	60.84	59.90	59.14	61.95	62.06	59.43	59.84	58.74	51.80	59.95	61.56	62.22	61.64	62.39	62.03
1488	372	16	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	60.04	61.75	59.80	59.81	61.99	50.27	59.95	61.77	59.55	59.66	60.77	88.17	59.69	62.41	62.64	91.16	60.96	58.88	60.43	62.26	62.46	63.04	59.57	62.34	62.61
1491	372	16	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	93.44	62.32	62.64	61.79	59.80	59.18	59.69	60.13	59.70	59.57	61.20	61.07	59.33	59.91	60.90	59.71	61.32	59.57	60.22	60.32	62.55	63.11	62.72	61.64	62.26
1500	372	16	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	59.09	60.32	59.82	62.64	59.19	60.37	59.53	61.76	51.03	59.61	60.67	61.26	60.80	60.72	62.64	59.23	60.95	59.56	62.97	62.50	62.72	61.39	62.17	62.80	61.33
1503	372	16	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	59.52	59.79	59.14	59.95	59.80	60.87	52.36	62.97	59.37	62.03	62.97	62.39	60.97	61.21	62.32	59.61	61.70	59.71	59.98	59.62	60.25	62.75	62.48	62.72	62.20
1552	372	16	0.1	16	0.00001	48	0.000000e+00	0.000000e+00	56.79	61.64	60.43	61.10	60.45	59.41	57.67	60.34	61.31	48.95	50.75	50.24	58.48	59.43	58.25	56.49	60.48	58.29	60.30	60.26	61.09	61.90	61.64	64.51	62.82
1555	372	16	0.1	16	0.00001	48	0.000000e+00	1.000000e-08	58.39	61.11	59.19	61.56	61.90	61.26	58.63	61.99	58.34	60.57	60.64	59.87	58.16	60.52	60.03	50.14	57.63	58.19	58.80	60.97	63.04	61.31	52.02	62.48	62.28
1564	372	16	0.1	16	0.00001	48	1.000000e-08	0.000000e+00	50.45	57.42	56.67	55.72	60.53	58.33	57.19	53.99	49.03	57.48	58.32	59.64	57.99	59.86	49.04	57.25	58.73	58.26	58.38	58.04	50.77	59.93	61.40	60.17	60.77
1567	372	16	0.1	16	0.00001	48	1.000000e-08	1.000000e-08	57.48	60.80	60.90	59.45	61.99	56.69	58.34	60.69	60.73	51.33	57.89	50.39	58.36	60.96	60.44	49.33	58.31	58.67	60.63	58.36	61.12	61.74	61.14	62.99	60.68

Model performance on validation data (T-test)

Statistical Significance:

Using the global mean maximum row as the reference, we perform the right-tailed t-test to identify significant hyperparameters.

pvalues = []
pvalues_real = []

for _,index in enumerate(dt_valid_sub.index):
  _, p_value = stats.ttest_ind(max_row[:-1], dt_valid_sub.loc[index][:-1], alternative='greater')
  pvalues_real.append(p_value)
  pvalues.append((p_value >= 0.05)*1)

  

dt_valid_sub['pvalues'] = pvalues_real
dt_valid_sub['sig'] = pvalues


dt_valid_sub1 = dt_valid_sub.query('sig == 1')


max_indexx = dt_valid_sub1.iloc[:, :-3].mean().idxmax()

#print(sum(pvalues),max_indexx)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_zero	mAP_valid_zero_2	mAP_valid_zero_3	mAP_valid_zero_4	mAP_valid_zero_5	mAP_valid_zero_6	mAP_valid_zero_7	mAP_valid_zero_8	mAP_valid_zero_9	mAP_valid_zero_10	mAP_valid_zero_11	mAP_valid_zero_12	mAP_valid_zero_13	mAP_valid_zero_14	mAP_valid_zero_15	mAP_valid_zero_16	mAP_valid_zero_17	mAP_valid_zero_18	mAP_valid_zero_19	mAP_valid_zero_20	mAP_valid_zero_21	mAP_valid_zero_22	mAP_valid_zero_23	mAP_valid_zero_25	mAP_valid_zero_26
1232	372	16	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	83.53	88.44	84.88	85.08	85.08	89.43	84.85	90.33	84.04	87.40	86.57	87.65	85.39	88.88	88.49	87.10	87.04	87.07	88.05	90.21	89.57	89.90	86.93	88.73	88.68
1235	372	16	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	88.08	88.34	83.96	87.54	88.65	88.35	85.47	85.72	84.54	88.33	87.71	86.85	82.69	87.69	88.06	86.95	87.56	88.23	87.94	88.60	88.42	89.57	88.92	88.46	89.61
1244	372	16	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	86.99	56.21	89.61	87.22	85.27	86.52	84.44	86.52	84.84	87.05	89.35	88.93	86.18	86.93	90.46	88.28	90.32	87.66	87.77	92.33	87.06	87.15	89.89	89.03	87.86
1247	372	16	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	84.47	83.01	85.60	85.18	85.28	87.60	85.33	86.50	87.33	87.77	89.77	90.62	84.97	87.59	85.17	86.55	88.49	85.67	90.47	91.00	86.41	90.04	88.63	89.31	91.02
659	372	16	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	84.48	86.20	85.22	84.23	82.40	85.49	83.94	86.14	84.96	90.00	85.69	89.30	88.31	90.67	86.05	87.08	86.65	85.99	88.84	91.57	88.34	89.37	88.76	90.95	87.33
668	372	16	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	84.83	83.87	85.88	85.94	84.20	86.71	83.54	88.89	86.68	85.46	90.01	86.19	86.43	86.14	88.75	88.74	89.55	86.60	90.65	87.92	89.38	89.22	88.77	90.98	88.30
671	372	16	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	83.26	86.21	86.70	86.64	86.79	88.41	83.54	87.94	85.52	87.25	86.95	88.36	86.00	87.76	87.40	88.76	85.35	87.11	89.34	87.46	87.33	89.84	90.51	88.31	88.02
80	372	16	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	84.63	88.42	83.59	84.03	87.12	84.90	84.35	87.86	87.60	85.67	90.03	84.55	87.08	43.72	90.41	86.71	86.93	88.59	86.21	87.19	89.49	88.83	86.98	89.67	86.19
83	372	16	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	85.78	85.10	87.06	85.85	85.75	86.08	87.05	86.50	88.49	85.51	89.27	85.33	88.33	88.43	89.13	88.14	87.98	89.70	90.01	87.96	89.56	88.86	89.76	89.34	88.81
92	372	16	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	85.33	55.49	85.95	86.99	82.75	86.35	83.79	88.80	90.37	83.49	87.88	92.20	87.57	85.43	90.77	87.66	89.01	89.83	88.83	88.65	87.71	88.24	89.60	89.37	89.74
95	372	16	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	84.39	85.29	84.06	88.72	85.74	90.04	87.22	87.87	86.75	87.21	84.39	52.60	85.86	86.53	89.72	87.72	84.87	86.44	88.54	89.61	87.53	88.71	88.63	87.39	91.23
1308	372	16	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	84.38	84.40	86.42	87.16	83.94	89.47	85.43	86.50	87.49	88.84	92.23	85.77	87.70	84.63	84.99	88.64	88.38	85.77	88.99	87.56	87.51	87.63	86.90	94.54	89.58
1311	372	16	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	84.86	88.25	85.90	83.87	84.58	85.85	85.17	88.22	85.54	86.28	91.55	88.28	88.10	85.41	86.97	87.23	86.29	87.53	91.86	87.80	89.01	89.03	87.46	88.34	90.09
720	372	16	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	84.04	86.42	83.62	87.03	84.32	86.52	86.53	85.36	88.58	84.96	87.46	88.42	86.99	86.31	87.64	86.45	90.35	88.16	90.52	91.09	94.45	87.99	88.48	88.75	86.83
723	372	16	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	87.04	89.27	86.62	86.82	86.33	91.31	84.03	89.00	88.66	86.85	90.99	87.54	87.29	87.78	89.05	86.66	87.18	88.10	88.82	93.96	91.12	89.24	87.17	55.98	90.02
732	372	16	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	82.65	85.25	84.79	43.72	83.75	87.82	84.32	88.47	84.22	84.95	90.98	87.10	88.02	88.46	85.99	88.04	86.45	86.56	85.51	87.03	88.77	89.08	87.96	89.26	88.93
735	372	16	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	83.55	85.74	84.63	84.88	82.47	88.00	84.12	87.47	86.10	87.44	90.31	87.43	89.65	85.66	88.47	85.50	90.54	84.64	89.04	89.56	90.07	89.50	88.34	90.37	91.10
144	372	16	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	83.67	84.79	86.13	86.58	85.21	88.42	85.83	88.01	87.18	87.61	88.45	88.65	86.86	87.87	88.30	55.25	43.72	88.15	87.41	88.98	88.66	88.26	88.39	91.72	89.32
147	372	16	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	86.13	85.61	83.84	89.78	86.49	84.58	86.42	87.88	84.90	88.74	87.54	91.68	87.17	86.77	43.72	86.43	43.72	89.07	89.56	90.28	88.86	88.90	88.16	90.41	88.53
156	372	16	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	83.43	85.49	84.57	84.11	88.36	88.90	85.41	84.37	85.14	85.20	91.59	89.12	85.76	86.73	87.82	88.02	87.14	87.13	88.27	87.58	88.95	89.00	89.24	90.10	90.54
159	372	16	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	86.45	84.66	84.54	83.12	86.23	84.71	86.29	86.77	87.14	88.42	87.90	90.15	88.27	85.89	87.39	87.71	87.08	87.84	86.64	92.44	89.17	90.39	86.94	90.74	89.45

Size of the All data:  (100, 28)
Size of the Sig data:  (21, 28)

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params1 = dt_valid.loc[dt_valid_sub1.index][['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
optimal_params = optimal_params1.reset_index()
optimal_params.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_1 = pd.merge(optimal_params, dt_test, on=['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg'])
test_data_1_sub = test_data_1[list(test_data_1.columns[test_data_1.columns.str.startswith('mAP_test')])]

Equivalent test data

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_zero	mAP_test_zero_2	mAP_test_zero_3	mAP_test_zero_4	mAP_test_zero_5	mAP_test_zero_6	mAP_test_zero_7	mAP_test_zero_8	mAP_test_zero_9	mAP_test_zero_10	mAP_test_zero_11	mAP_test_zero_12	mAP_test_zero_13	mAP_test_zero_14	mAP_test_zero_15	mAP_test_zero_16	mAP_test_zero_17	mAP_test_zero_18	mAP_test_zero_19	mAP_test_zero_20	mAP_test_zero_21	mAP_test_zero_22	mAP_test_zero_23	mAP_test_zero_25	mAP_test_zero_26
9	372	16	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	91.98	90.81	94.19	90.04	89.78	91.23	93.26	91.17	93.62	89.31	93.19	87.83	94.32	92.77	70.81	94.43	90.43	90.22	90.41	91.43	82.66	92.47	92.91	91.48	90.61
3	372	16	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	89.49	86.62	93.62	67.86	93.02	92.15	93.44	90.97	90.61	94.83	92.54	89.83	93.51	91.92	89.66	93.63	96.14	94.86	94.15	91.40	93.39	93.46	93.06	59.77	91.23
14	372	16	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	91.54	92.46	93.30	91.73	95.96	93.78	93.88	93.63	89.45	93.43	91.94	90.88	90.20	91.56	90.04	94.46	92.45	91.51	93.73	91.80	92.01	86.17	92.01	91.54	92.13
6	372	16	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	94.20	95.16	93.76	92.47	94.85	94.95	91.46	93.41	94.73	94.36	93.38	91.38	90.08	89.94	93.46	93.53	95.17	92.90	93.26	93.37	91.25	92.30	93.47	91.38	88.62
12	372	16	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	92.94	93.02	94.02	91.73	93.67	91.16	93.56	92.34	93.53	94.10	91.68	93.23	90.68	93.24	85.87	93.77	91.67	94.26	91.34	93.87	94.04	92.40	91.06	59.76	92.98
7	372	16	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	90.09	89.83	94.95	92.58	94.22	92.98	93.77	59.19	89.68	94.24	93.43	90.63	95.19	92.18	93.14	93.84	91.38	91.87	92.23	94.45	93.09	92.76	91.65	60.71	93.31
11	372	16	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	94.85	93.50	94.33	93.60	88.74	92.37	95.52	91.14	93.30	93.32	93.02	91.58	94.45	94.75	93.96	90.74	89.82	93.46	92.81	89.46	91.90	58.98	93.01	92.55	91.92
17	372	16	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	89.84	95.59	89.30	92.87	93.64	91.45	93.72	94.63	96.05	95.10	93.32	85.38	93.42	48.12	58.32	78.79	94.49	91.95	93.05	92.38	60.28	92.39	93.12	92.61	91.61
0	372	16	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	94.39	90.90	90.83	91.65	95.27	92.45	94.89	93.57	94.36	93.48	92.70	92.33	92.09	94.71	93.37	95.36	90.57	91.30	92.74	92.52	58.52	92.53	91.34	92.05	93.06
15	372	16	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	93.84	90.64	93.37	91.47	93.39	92.08	94.27	93.29	94.14	91.98	93.36	92.09	91.38	94.82	87.58	93.06	93.73	92.79	92.79	94.12	92.66	95.30	91.86	52.60	59.48
16	372	16	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	95.19	89.90	91.66	93.81	92.61	91.11	93.92	92.79	92.28	92.95	91.71	93.93	94.13	91.67	94.56	93.28	96.33	93.17	92.52	92.19	93.98	93.16	93.88	92.57	68.90
4	372	16	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	94.05	91.36	94.72	92.68	90.38	93.68	93.75	92.71	92.64	94.20	93.13	94.55	94.07	93.98	91.67	92.87	92.13	94.84	94.51	92.39	92.34	91.87	89.13	62.26	87.81
8	372	16	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	93.36	92.41	91.80	89.90	90.39	92.37	94.18	94.33	93.04	93.96	91.64	93.59	92.92	95.46	92.38	92.40	90.95	92.07	92.96	93.30	91.36	95.07	88.88	93.09	92.51
1	372	16	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	92.67	94.38	93.64	94.11	94.49	93.21	91.39	89.39	93.59	93.39	94.72	93.42	94.50	87.86	93.49	91.52	91.68	93.76	91.13	92.58	61.03	93.21	94.43	93.94	89.78
13	372	16	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	95.18	92.56	88.21	90.99	91.86	92.64	94.47	93.97	92.58	92.19	92.14	91.85	92.77	94.48	51.17	94.34	94.34	92.46	92.37	93.06	61.74	92.58	91.18	94.59	92.67
18	372	16	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	92.88	91.41	90.16	48.12	93.75	93.14	93.74	91.06	93.60	92.77	93.67	90.63	92.24	90.56	95.45	87.41	91.94	92.89	90.92	91.74	91.13	91.46	93.77	92.72	91.90
5	372	16	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	95.10	93.45	95.77	91.04	89.61	92.23	93.37	92.65	92.79	93.95	89.07	93.26	90.39	92.84	91.41	93.56	90.51	90.42	92.16	92.91	92.24	92.53	61.10	92.86	70.81
19	372	16	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	94.08	92.56	95.00	91.98	95.50	89.95	92.75	93.10	94.56	93.18	92.63	92.37	94.71	94.65	88.54	92.71	48.12	93.22	89.43	91.09	91.74	92.48	92.68	56.57	92.61
20	372	16	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	94.33	96.36	91.19	93.37	95.37	92.15	95.07	93.06	93.12	93.08	93.84	92.27	91.78	86.90	48.12	95.65	48.12	91.03	92.12	92.25	93.78	93.49	92.86	91.47	92.20
10	372	16	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	91.04	94.11	90.14	92.09	94.27	93.11	93.46	91.67	95.06	94.48	92.71	92.29	94.73	92.03	94.63	94.35	93.27	94.86	92.34	92.36	93.98	94.02	94.16	92.60	90.30
2	372	16	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	92.42	88.17	93.81	88.32	93.18	93.06	92.77	93.30	94.09	93.37	91.78	92.68	91.98	95.89	94.92	93.97	92.23	93.10	93.33	91.20	93.22	92.72	92.72	90.34	93.27

Size of the test data:  (21, 33)

Difference

diff = dt[['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
for i, cols in enumerate(zip(dt.columns,test_data_1.columns)):   
   diff[f'col_diff_{i}'] = (np.array(dt[cols[0]]) - np.array(test_data_1[cols[1]]))*-1

# Function to count negatives in a row
def count_negatives(row):
    return sum(1 for value in row if isinstance(value, (int, float)) and value < 0)

diff['negative_count'] = diff.apply(count_negatives, axis=1)
diff.sort_values(by =["negative_count"],inplace = True, ascending=False)

html_table = diff.to_html(index=True)

# Wrap in a scrollable div
scrollable_table = f"""
<div style="height: 400px; width: 100%; overflow-x: auto; overflow-y: auto;">
    {html_table}
</div>
"""
# Display the scrollable table
display(HTML(scrollable_table))

# 
# show(diff)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	col_diff_0	col_diff_1	col_diff_2	col_diff_3	col_diff_4	col_diff_5	col_diff_6	col_diff_7	col_diff_8	col_diff_9	col_diff_10	col_diff_11	col_diff_12	col_diff_13	col_diff_14	col_diff_15	col_diff_16	col_diff_17	col_diff_18	col_diff_19	col_diff_20	col_diff_21	col_diff_22	col_diff_23	col_diff_24	col_diff_25	col_diff_26	col_diff_27	col_diff_28	col_diff_29	col_diff_30	col_diff_31	col_diff_32	negative_count
92	372	16	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.51	35.15	7.42	4.48	10.64	5.73	10.48	4.49	3.77	8.49	5.48	-0.11	3.81	9.39	-3.19	5.40	4.72	2.96	3.96	5.47	4.95	7.06	2.26	-36.77	-30.26	4
735	372	16	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	11.55	7.71	11.14	6.16	7.14	4.23	9.25	5.18	6.69	6.51	-1.24	5.83	0.74	7.18	2.94	8.06	-0.03	5.78	3.12	3.35	2.17	3.03	-27.24	2.49	-20.29	4
1244	372	16	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	4.55	36.25	3.69	4.51	10.69	7.26	9.44	7.11	4.61	6.38	2.59	1.95	4.02	4.63	-0.42	6.18	2.13	3.85	5.96	-0.53	4.95	-0.98	2.12	2.51	4.27	3
723	372	16	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.14	3.29	1.59	4.17	5.53	1.33	10.44	4.97	3.92	5.34	1.15	4.31	5.48	6.70	-37.88	7.68	7.16	4.36	3.55	-0.90	-29.38	3.34	4.01	38.61	2.65	3
1235	372	16	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	1.41	-1.72	9.66	-19.68	4.37	3.80	7.97	5.25	6.07	6.50	4.83	2.98	10.82	4.23	1.60	6.68	8.58	6.63	6.21	2.80	4.97	3.89	4.14	-28.69	1.62	3
80	372	16	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.21	7.17	5.71	8.84	6.52	6.55	9.37	6.77	8.45	9.43	3.29	0.83	6.34	4.40	-32.09	-7.92	7.56	3.36	6.84	5.19	-29.21	3.56	6.14	2.94	5.42	3
1232	372	16	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.45	2.37	9.31	4.96	4.70	1.80	8.41	0.84	9.58	1.91	6.62	0.18	8.93	3.89	-17.68	7.33	3.39	3.15	2.36	1.22	-6.91	2.57	5.98	2.75	1.93	2
1308	372	16	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.67	6.96	8.30	5.52	6.44	4.21	8.32	6.21	5.15	5.36	0.90	8.78	6.37	9.35	6.68	4.23	3.75	9.07	5.52	4.83	4.83	4.24	2.23	-32.28	-1.77	2
159	372	16	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.97	3.51	9.27	5.20	6.95	8.35	6.48	6.53	6.95	4.95	3.88	2.53	3.71	10.00	7.53	6.26	5.15	5.26	6.69	-1.24	4.05	2.33	5.78	-0.40	3.82	2
668	372	16	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.26	5.96	9.07	6.64	10.02	6.27	10.23	-29.70	3.00	8.78	3.42	4.44	8.76	6.04	4.39	5.10	1.83	5.27	1.58	6.53	3.71	3.54	2.88	-30.27	5.01	2
659	372	16	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.46	6.82	8.80	7.50	11.27	5.67	9.62	6.20	8.57	4.10	5.99	3.93	2.37	2.57	-0.18	6.69	5.02	8.27	2.50	2.30	5.70	3.03	2.30	-31.19	5.65	2
83	372	16	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.61	5.80	3.77	5.80	9.52	6.37	7.84	7.07	5.87	7.97	3.43	7.00	3.76	6.28	4.24	7.22	2.59	1.60	2.73	4.56	-31.04	3.67	1.58	2.71	4.25	1
671	372	16	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	11.59	7.29	7.63	6.96	1.95	3.96	11.98	3.20	7.78	6.07	6.07	3.22	8.45	6.99	6.56	1.98	4.47	6.35	3.47	2.00	4.57	-30.86	2.50	4.24	3.90	1
720	372	16	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.63	7.96	10.02	7.08	10.17	6.69	4.86	4.03	5.01	8.43	7.26	5.00	7.51	1.55	5.85	5.07	1.33	5.60	0.61	1.49	-33.42	5.22	5.95	5.19	2.95	1
732	372	16	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	10.23	6.16	5.37	4.40	10.00	5.32	9.42	2.59	9.38	7.82	2.69	3.53	4.22	2.10	9.46	-0.63	5.49	6.33	5.41	4.71	2.36	2.38	5.81	3.46	2.97	1
1247	372	16	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.73	12.15	8.16	7.29	9.57	7.35	6.13	6.91	7.40	6.59	3.61	0.76	5.11	2.35	8.29	6.98	6.68	7.23	2.79	2.37	4.84	2.26	4.84	2.07	-2.40	1
144	372	16	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	10.41	7.77	8.87	5.40	10.29	1.53	6.92	5.09	7.38	5.57	4.18	3.72	7.85	6.78	0.24	37.46	4.40	5.07	2.02	2.11	3.08	4.22	4.29	-35.15	3.29	1
156	372	16	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.61	8.62	5.57	7.98	5.91	4.21	8.05	7.30	9.92	9.28	1.12	3.17	8.97	5.30	6.81	6.33	6.13	7.73	4.07	4.78	5.03	5.02	4.92	2.50	-0.24	1
95	372	16	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	10.80	4.61	7.60	5.09	6.87	1.07	6.70	4.92	5.53	5.74	7.32	41.33	8.27	5.14	4.84	5.56	11.46	6.73	3.98	2.58	6.45	4.45	5.25	5.18	-22.33	1
1311	372	16	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.50	4.16	5.90	6.03	5.81	6.52	9.01	6.11	7.50	7.68	0.09	5.31	4.82	10.05	5.41	5.17	4.66	4.54	1.10	5.50	2.35	6.04	1.42	4.75	2.42	0
147	372	16	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.20	10.75	7.35	3.59	8.88	7.57	8.65	5.18	8.22	4.34	6.30	0.59	4.61	0.13	4.40	9.22	4.40	1.96	2.56	1.97	4.92	4.59	4.70	1.06	3.67	0

Model performance on valid data

Average & Std Deviation of the Significant rows:

	mean	std
mAP_valid_zero_20	89.465714	1.991483
mAP_valid_zero_26	89.103810	1.376704
mAP_valid_zero_22	88.988095	0.804323
mAP_valid_zero_21	88.922381	1.681303
mAP_valid_zero_11	88.886667	2.086737
mAP_valid_zero_19	88.727143	1.544235
mAP_valid_zero_23	88.400952	1.061291
mAP_valid_zero_25	88.178571	7.532805
mAP_valid_zero_18	87.420952	1.337467
mAP_valid_zero_6	87.402857	1.854880
mAP_valid_zero_8	87.386667	1.411097
mAP_valid_zero_13	86.886667	1.502975
mAP_valid_zero_10	86.877619	1.599021
mAP_valid_zero_12	86.510476	8.012257
mAP_valid_zero_9	86.479524	1.709335
mAP_valid_zero_15	85.940476	9.809429
mAP_valid_zero_16	85.886667	7.073603
mAP_valid_zero_3	85.408095	1.440426
mAP_valid_zero_5	85.271905	1.728585
mAP_valid_zero_7	85.098571	1.143155
mAP_valid_zero_14	85.013333	9.564064
mAP_valid_zero	84.855714	1.427055
mAP_valid_zero_4	84.023333	9.386467
mAP_valid_zero_17	83.552381	13.333567
mAP_valid_zero_2	83.164762	9.231600

Model performance on Test data

We then apply the hyperparameters to the test set and average the results.

test_data_1['average_map'] = test_data_1_sub.apply(np.mean, axis=1)

test_data_1.sort_values(by=['average_map'], ascending= False,inplace=True)

	mean	std
mAP_test_zero_7	93.649524	1.008928
mAP_test_zero_10	93.412857	1.233787
mAP_test_zero_9	93.181905	1.647725
mAP_test_zero_5	93.045238	2.135703
mAP_test_zero	93.021905	1.786991
mAP_test_zero_13	92.835238	1.647351
mAP_test_zero_3	92.750952	2.094173
mAP_test_zero_18	92.711429	1.378370
mAP_test_zero_11	92.647619	1.155318
mAP_test_zero_16	92.555714	3.608916
mAP_test_zero_6	92.440476	1.101919
mAP_test_zero_19	92.395238	1.223747
mAP_test_zero_20	92.374762	1.149059
mAP_test_zero_2	92.152381	2.407048
mAP_test_zero_12	91.714286	2.104104
mAP_test_zero_8	91.017619	7.406802
mAP_test_zero_22	91.016667	7.545582
average_map	90.937048	1.388388
mAP_test_zero_23	90.870476	6.976633
mAP_test_zero_14	90.491905	9.993483
mAP_test_zero_4	88.686190	10.760625
mAP_test_zero_17	88.355714	13.504975
mAP_test_zero_26	87.986190	9.348518
mAP_test_zero_21	86.016190	12.960290
mAP_test_zero_15	85.359524	14.788780
mAP_test_zero_25	82.736190	15.756536

Model performance on validation data (Mann-Whitney U Test)

Statistical Significance:

Using the global mean maximum row as the reference, we perform the right-tailed t-test to identify significant hyperparameters.

pvalues_mw = []
pvalues_real_mw = []

for _,index in enumerate(dt_valid_sub.index):

  _, p_value_mw = stats.mannwhitneyu(max_row[:-1], dt_valid_sub.loc[index][:-1], alternative='greater')#, method = 'asymptotic')
  pvalues_real_mw.append(p_value_mw)
  pvalues_mw.append((p_value_mw >= 0.05)*1)
  

dt_valid_sub_mw['pvalues_mw'] = pvalues_real_mw
dt_valid_sub_mw['sig_mw'] = pvalues_mw

dt_valid_sub_mw1 = dt_valid_sub_mw.query('sig_mw == 1')


max_indexx = dt_valid_sub_mw1.iloc[:, :-3].mean().idxmax()

#print(sum(pvalues),max_indexx)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_zero	mAP_valid_zero_2	mAP_valid_zero_3	mAP_valid_zero_4	mAP_valid_zero_5	mAP_valid_zero_6	mAP_valid_zero_7	mAP_valid_zero_8	mAP_valid_zero_9	mAP_valid_zero_10	mAP_valid_zero_11	mAP_valid_zero_12	mAP_valid_zero_13	mAP_valid_zero_14	mAP_valid_zero_15	mAP_valid_zero_16	mAP_valid_zero_17	mAP_valid_zero_18	mAP_valid_zero_19	mAP_valid_zero_20	mAP_valid_zero_21	mAP_valid_zero_22	mAP_valid_zero_23	mAP_valid_zero_25	mAP_valid_zero_26
607	372	16	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	67.25	69.05	87.80	88.19	88.71	85.38	83.67	90.58	90.27	90.32	88.71	64.42	87.92	88.57	84.30	83.58	87.51	85.72	81.95	72.89	90.53	88.11	88.10	89.10	87.69
1232	372	16	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	83.53	88.44	84.88	85.08	85.08	89.43	84.85	90.33	84.04	87.40	86.57	87.65	85.39	88.88	88.49	87.10	87.04	87.07	88.05	90.21	89.57	89.90	86.93	88.73	88.68
1235	372	16	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	88.08	88.34	83.96	87.54	88.65	88.35	85.47	85.72	84.54	88.33	87.71	86.85	82.69	87.69	88.06	86.95	87.56	88.23	87.94	88.60	88.42	89.57	88.92	88.46	89.61
1244	372	16	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	86.99	56.21	89.61	87.22	85.27	86.52	84.44	86.52	84.84	87.05	89.35	88.93	86.18	86.93	90.46	88.28	90.32	87.66	87.77	92.33	87.06	87.15	89.89	89.03	87.86
1247	372	16	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	84.47	83.01	85.60	85.18	85.28	87.60	85.33	86.50	87.33	87.77	89.77	90.62	84.97	87.59	85.17	86.55	88.49	85.67	90.47	91.00	86.41	90.04	88.63	89.31	91.02
1424	372	16	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	89.43	59.37	60.86	90.31	60.29	91.97	89.47	60.36	89.05	59.76	92.47	57.77	92.39	91.04	60.33	92.93	83.96	90.55	91.55	90.95	91.35	57.92	57.50	58.23	59.09
1427	372	16	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	88.66	62.08	86.90	61.21	61.26	90.62	88.46	60.64	90.54	61.59	91.46	91.96	90.92	91.47	62.54	91.10	60.73	91.16	92.97	58.62	59.00	59.07	56.94	59.82	59.26
1436	372	16	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	88.43	61.33	88.44	88.77	88.55	89.92	87.41	61.70	62.12	91.44	91.11	90.45	87.65	91.46	60.53	89.66	61.29	91.17	58.10	91.29	58.26	57.54	61.57	61.55	59.67
1439	372	16	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	88.21	89.50	88.75	59.01	91.13	91.66	91.21	60.61	90.19	58.20	89.71	92.18	90.96	59.26	60.79	91.55	90.78	91.69	89.00	92.93	60.76	57.98	59.35	57.82	57.90
659	372	16	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	84.48	86.20	85.22	84.23	82.40	85.49	83.94	86.14	84.96	90.00	85.69	89.30	88.31	90.67	86.05	87.08	86.65	85.99	88.84	91.57	88.34	89.37	88.76	90.95	87.33
668	372	16	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	84.83	83.87	85.88	85.94	84.20	86.71	83.54	88.89	86.68	85.46	90.01	86.19	86.43	86.14	88.75	88.74	89.55	86.60	90.65	87.92	89.38	89.22	88.77	90.98	88.30
671	372	16	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	83.26	86.21	86.70	86.64	86.79	88.41	83.54	87.94	85.52	87.25	86.95	88.36	86.00	87.76	87.40	88.76	85.35	87.11	89.34	87.46	87.33	89.84	90.51	88.31	88.02
848	372	16	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	88.80	60.46	88.99	62.31	61.12	92.25	89.45	90.54	90.01	55.67	91.68	59.18	90.55	91.72	60.28	89.32	60.38	90.76	90.93	91.20	56.95	60.00	59.84	58.69	58.10
851	372	16	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	89.05	61.41	61.72	61.05	89.20	92.43	89.23	90.97	90.68	90.06	92.05	58.77	91.68	90.59	58.42	92.73	58.84	89.71	91.67	92.46	58.02	58.82	90.25	57.72	57.60
863	372	16	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	87.15	89.51	87.86	60.59	89.11	89.70	89.77	60.72	91.09	91.72	56.53	93.59	89.17	59.55	60.05	91.21	60.87	91.19	60.25	88.59	58.13	57.64	83.08	60.50	58.19
1043	372	16	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	88.74	61.30	56.02	60.76	90.23	91.55	90.83	89.82	87.79	90.48	92.69	92.21	92.17	90.13	60.65	90.16	91.23	60.76	91.68	58.62	60.26	59.37	91.31	59.32	59.55
1052	372	16	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	89.22	56.29	89.10	89.55	89.98	92.03	88.29	86.85	90.51	91.39	91.67	91.15	85.73	91.43	56.79	91.73	92.20	91.14	91.67	93.30	57.33	59.70	89.95	59.53	58.52
83	372	16	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	85.78	85.10	87.06	85.85	85.75	86.08	87.05	86.50	88.49	85.51	89.27	85.33	88.33	88.43	89.13	88.14	87.98	89.70	90.01	87.96	89.56	88.86	89.76	89.34	88.81
92	372	16	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	85.33	55.49	85.95	86.99	82.75	86.35	83.79	88.80	90.37	83.49	87.88	92.20	87.57	85.43	90.77	87.66	89.01	89.83	88.83	88.65	87.71	88.24	89.60	89.37	89.74
95	372	16	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	84.39	85.29	84.06	88.72	85.74	90.04	87.22	87.87	86.75	87.21	84.39	52.60	85.86	86.53	89.72	87.72	84.87	86.44	88.54	89.61	87.53	88.71	88.63	87.39	91.23
284	372	16	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	88.12	56.48	90.11	61.64	62.69	90.30	89.46	59.56	91.30	60.93	91.97	92.55	89.70	90.04	61.86	91.19	55.31	91.03	91.43	92.23	59.09	58.29	89.92	58.82	58.82
287	372	16	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	88.39	90.65	88.41	61.00	88.52	91.99	89.35	59.44	91.54	91.49	90.93	58.79	90.42	91.45	92.63	82.72	58.98	90.62	58.93	91.19	56.88	57.78	59.02	60.00	57.57
479	372	16	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	87.95	61.87	89.97	61.10	88.97	91.42	91.37	59.53	56.52	90.71	92.44	57.58	90.43	58.92	89.22	91.93	91.73	83.67	92.46	90.16	88.59	55.64	89.38	57.92	56.12
1308	372	16	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	84.38	84.40	86.42	87.16	83.94	89.47	85.43	86.50	87.49	88.84	92.23	85.77	87.70	84.63	84.99	88.64	88.38	85.77	88.99	87.56	87.51	87.63	86.90	94.54	89.58
1311	372	16	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	84.86	88.25	85.90	83.87	84.58	85.85	85.17	88.22	85.54	86.28	91.55	88.28	88.10	85.41	86.97	87.23	86.29	87.53	91.86	87.80	89.01	89.03	87.46	88.34	90.09
1488	372	16	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	89.14	61.27	88.17	89.93	61.05	87.32	90.16	60.27	91.91	88.92	89.61	81.34	89.91	60.86	58.94	85.71	57.72	92.71	89.90	57.22	57.08	56.61	89.86	57.30	55.88
1491	372	16	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	84.26	61.17	59.80	60.26	88.17	91.72	89.41	90.90	91.95	89.42	88.79	89.80	90.45	90.92	59.30	91.43	58.08	90.67	91.19	90.92	58.06	54.86	58.52	58.95	59.23
1500	372	16	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	90.54	89.65	88.40	60.17	88.71	91.64	89.95	59.78	87.36	90.44	90.13	88.29	87.89	59.52	59.36	92.22	58.65	90.68	56.51	59.00	55.90	59.04	57.28	56.17	59.28
1503	372	16	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	89.43	89.17	88.15	89.16	89.32	90.86	85.88	58.54	90.37	59.20	56.75	59.20	88.27	61.32	59.48	92.18	59.51	90.31	93.47	92.35	89.52	58.12	57.56	56.73	58.39
720	372	16	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	84.04	86.42	83.62	87.03	84.32	86.52	86.53	85.36	88.58	84.96	87.46	88.42	86.99	86.31	87.64	86.45	90.35	88.16	90.52	91.09	94.45	87.99	88.48	88.75	86.83
723	372	16	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	87.04	89.27	86.62	86.82	86.33	91.31	84.03	89.00	88.66	86.85	90.99	87.54	87.29	87.78	89.05	86.66	87.18	88.10	88.82	93.96	91.12	89.24	87.17	55.98	90.02
735	372	16	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	83.55	85.74	84.63	84.88	82.47	88.00	84.12	87.47	86.10	87.44	90.31	87.43	89.65	85.66	88.47	85.50	90.54	84.64	89.04	89.56	90.07	89.50	88.34	90.37	91.10
912	372	16	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	88.66	62.10	87.68	89.15	89.47	92.36	86.60	89.37	90.15	82.22	56.01	59.21	91.43	91.94	59.78	90.66	90.92	91.72	92.17	58.52	55.65	56.58	57.43	59.02	56.83
924	372	16	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	90.20	88.68	63.43	88.68	89.20	90.89	85.79	58.90	89.17	90.71	91.40	56.33	89.36	90.71	60.14	91.19	59.64	89.66	90.66	59.17	56.23	56.00	57.16	56.15	58.95
927	372	16	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	88.46	87.09	89.66	60.96	89.22	91.97	85.83	87.38	90.35	91.21	58.52	60.25	87.04	89.10	60.31	88.72	59.70	88.67	90.39	91.17	58.57	57.66	88.05	57.70	59.64
1107	372	16	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	87.39	89.69	88.72	61.07	89.93	92.22	86.84	59.25	90.46	59.42	90.13	58.04	90.35	91.92	88.77	91.27	60.30	90.42	90.62	90.63	57.06	57.47	57.67	59.67	57.11
1119	372	16	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	87.26	61.63	89.40	90.71	90.93	90.38	88.92	60.44	89.36	88.89	89.74	91.42	91.67	90.17	59.04	86.90	92.17	88.38	90.63	57.64	56.45	57.74	57.84	56.86	87.26
144	372	16	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	83.67	84.79	86.13	86.58	85.21	88.42	85.83	88.01	87.18	87.61	88.45	88.65	86.86	87.87	88.30	55.25	43.72	88.15	87.41	88.98	88.66	88.26	88.39	91.72	89.32
147	372	16	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	86.13	85.61	83.84	89.78	86.49	84.58	86.42	87.88	84.90	88.74	87.54	91.68	87.17	86.77	43.72	86.43	43.72	89.07	89.56	90.28	88.86	88.90	88.16	90.41	88.53
156	372	16	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	83.43	85.49	84.57	84.11	88.36	88.90	85.41	84.37	85.14	85.20	91.59	89.12	85.76	86.73	87.82	88.02	87.14	87.13	88.27	87.58	88.95	89.00	89.24	90.10	90.54
159	372	16	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	86.45	84.66	84.54	83.12	86.23	84.71	86.29	86.77	87.14	88.42	87.90	90.15	88.27	85.89	87.39	87.71	87.08	87.84	86.64	92.44	89.17	90.39	86.94	90.74	89.45
336	372	16	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	87.64	61.05	87.87	60.19	90.47	91.06	89.66	59.61	90.93	59.59	92.06	89.31	87.80	58.83	59.21	89.18	58.31	91.43	89.09	91.96	58.79	58.31	59.19	59.65	57.70
339	372	16	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	87.41	90.69	86.65	60.40	87.11	92.18	85.83	59.96	90.15	89.16	89.54	58.52	84.51	57.93	60.38	90.66	91.71	90.19	57.70	58.87	58.25	56.58	90.42	56.61	57.64
348	372	16	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	89.66	60.44	88.66	60.41	89.72	93.22	89.91	60.42	87.90	90.94	58.35	93.18	86.83	91.42	89.92	91.42	59.48	92.97	89.16	92.17	55.85	57.59	92.46	54.75	57.24
351	372	16	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	87.64	59.77	87.64	61.82	89.67	89.14	88.40	89.16	89.93	89.90	91.68	89.03	91.65	87.55	43.72	86.68	90.68	90.42	57.16	89.15	88.62	56.83	58.83	59.35	60.03
528	372	16	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	90.17	90.92	59.26	90.49	88.40	92.18	89.90	90.12	90.70	89.42	92.47	56.30	87.56	60.02	58.48	90.37	59.01	92.74	57.08	58.63	58.60	56.73	56.76	57.91	56.86
531	372	16	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	88.02	88.67	88.41	90.09	89.68	89.27	90.24	59.27	88.01	89.41	89.78	58.16	90.36	90.97	60.61	90.91	59.14	90.90	92.44	90.36	89.38	57.78	58.13	57.38	58.39

Size of the All data:  (100, 28)
Size of the Sig data:  (47, 33)

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params_mw1 = dt_valid.loc[dt_valid_sub_mw1.index][['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
optimal_params_mw = optimal_params_mw1.reset_index()
optimal_params_mw.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_mw1 = pd.merge(optimal_params_mw, dt_test, on=['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg'])
test_data_1_sub_mw = test_data_mw1[list(test_data_mw1.columns[test_data_mw1.columns.str.startswith('mAP_test')])]

Equivalent test data

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_zero	mAP_test_zero_2	mAP_test_zero_3	mAP_test_zero_4	mAP_test_zero_5	mAP_test_zero_6	mAP_test_zero_7	mAP_test_zero_8	mAP_test_zero_9	mAP_test_zero_10	mAP_test_zero_11	mAP_test_zero_12	mAP_test_zero_13	mAP_test_zero_14	mAP_test_zero_15	mAP_test_zero_16	mAP_test_zero_17	mAP_test_zero_18	mAP_test_zero_19	mAP_test_zero_20	mAP_test_zero_21	mAP_test_zero_22	mAP_test_zero_23	mAP_test_zero_25	mAP_test_zero_26
18	372	16	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	92.03	89.70	93.79	87.17	92.63	92.17	94.31	90.61	90.18	91.18	92.74	88.70	89.96	90.60	59.16	89.14	59.19	92.65	84.24	85.92	90.98	90.51	90.19	91.88	59.20
9	372	16	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	91.98	90.81	94.19	90.04	89.78	91.23	93.26	91.17	93.62	89.31	93.19	87.83	94.32	92.77	70.81	94.43	90.43	90.22	90.41	91.43	82.66	92.47	92.91	91.48	90.61
3	372	16	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	89.49	86.62	93.62	67.86	93.02	92.15	93.44	90.97	90.61	94.83	92.54	89.83	93.51	91.92	89.66	93.63	96.14	94.86	94.15	91.40	93.39	93.46	93.06	59.77	91.23
14	372	16	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	91.54	92.46	93.30	91.73	95.96	93.78	93.88	93.63	89.45	93.43	91.94	90.88	90.20	91.56	90.04	94.46	92.45	91.51	93.73	91.80	92.01	86.17	92.01	91.54	92.13
6	372	16	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	94.20	95.16	93.76	92.47	94.85	94.95	91.46	93.41	94.73	94.36	93.38	91.38	90.08	89.94	93.46	93.53	95.17	92.90	93.26	93.37	91.25	92.30	93.47	91.38	88.62
36	372	16	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	58.84	60.78	61.47	59.25	61.70	59.43	58.95	61.75	58.57	61.66	59.62	61.93	59.53	59.87	61.41	59.27	90.45	58.23	58.15	58.63	59.16	63.20	62.67	62.50	61.37
45	372	16	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	59.52	61.87	59.93	61.71	61.94	60.61	59.14	62.14	58.97	61.99	59.25	59.52	59.67	59.52	60.63	59.87	62.28	59.01	59.48	61.90	62.32	62.55	50.39	61.94	62.03
35	372	16	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	59.18	62.08	59.27	59.20	59.79	59.92	59.49	61.71	62.06	59.41	59.90	59.58	59.99	58.74	61.76	59.99	61.55	59.33	62.64	59.88	62.45	62.55	61.35	62.46	61.76
29	372	16	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	58.99	59.72	59.33	61.59	59.20	59.78	59.23	62.24	58.32	61.98	60.84	59.90	59.14	61.95	62.06	59.43	59.84	58.74	51.80	59.95	61.56	62.22	61.64	62.39	62.03
12	372	16	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	92.94	93.02	94.02	91.73	93.67	91.16	93.56	92.34	93.53	94.10	91.68	93.23	90.68	93.24	85.87	93.77	91.67	94.26	91.34	93.87	94.04	92.40	91.06	59.76	92.98
7	372	16	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	90.09	89.83	94.95	92.58	94.22	92.98	93.77	59.19	89.68	94.24	93.43	90.63	95.19	92.18	93.14	93.84	91.38	91.87	92.23	94.45	93.09	92.76	91.65	60.71	93.31
11	372	16	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	94.85	93.50	94.33	93.60	88.74	92.37	95.52	91.14	93.30	93.32	93.02	91.58	94.45	94.75	93.96	90.74	89.82	93.46	92.81	89.46	91.90	58.98	93.01	92.55	91.92
40	372	16	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	59.79	60.97	58.64	61.99	61.87	59.63	58.63	59.17	58.51	49.68	59.66	62.32	59.43	59.56	61.65	59.57	60.29	58.74	60.03	59.07	62.48	62.14	62.34	62.99	61.94
28	372	16	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	59.99	61.99	61.65	61.93	59.41	59.71	59.19	59.66	58.19	58.54	60.06	62.70	59.47	59.61	61.48	59.04	62.56	59.64	60.15	58.56	60.63	61.83	59.88	62.58	62.11
37	372	16	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	59.70	59.15	59.57	62.39	60.27	60.02	59.38	62.32	59.42	59.56	62.97	59.32	59.97	60.72	61.93	59.58	61.70	59.52	61.93	50.96	62.34	62.57	87.88	62.15	62.13
26	372	16	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	59.62	61.93	50.73	61.93	59.34	59.86	59.41	59.86	52.41	59.37	60.01	59.80	59.67	58.37	61.11	60.27	58.97	61.45	59.96	61.99	61.38	61.92	59.43	62.46	61.81
20	372	16	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	59.47	49.98	58.85	59.81	60.23	59.78	57.95	51.19	59.11	59.57	60.38	60.49	52.74	60.08	49.32	58.63	58.40	59.71	59.90	59.87	62.55	60.78	59.69	61.83	62.45
0	372	16	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	94.39	90.90	90.83	91.65	95.27	92.45	94.89	93.57	94.36	93.48	92.70	92.33	92.09	94.71	93.37	95.36	90.57	91.30	92.74	92.52	58.52	92.53	91.34	92.05	93.06
15	372	16	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	93.84	90.64	93.37	91.47	93.39	92.08	94.27	93.29	94.14	91.98	93.36	92.09	91.38	94.82	87.58	93.06	93.73	92.79	92.79	94.12	92.66	95.30	91.86	52.60	59.48
16	372	16	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	95.19	89.90	91.66	93.81	92.61	91.11	93.92	92.79	92.28	92.95	91.71	93.93	94.13	91.67	94.56	93.28	96.33	93.17	92.52	92.19	93.98	93.16	93.88	92.57	68.90
34	372	16	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	59.56	50.81	59.58	61.70	61.99	59.85	59.30	62.70	59.29	61.99	59.71	59.47	59.91	58.97	60.83	59.24	48.63	58.33	59.95	59.65	61.43	62.59	60.43	62.79	61.59
30	372	16	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	58.39	59.87	59.27	62.34	58.04	59.76	59.26	60.81	59.23	59.02	60.27	61.99	59.34	58.97	59.24	88.45	61.26	59.33	62.19	59.67	62.48	61.77	61.64	62.06	62.83
25	372	16	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	59.23	60.81	59.32	62.32	59.43	59.85	59.22	61.54	50.51	58.64	59.95	61.59	59.54	60.61	59.34	59.42	59.30	89.32	59.52	58.53	60.79	63.09	60.23	63.04	63.06
4	372	16	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	94.05	91.36	94.72	92.68	90.38	93.68	93.75	92.71	92.64	94.20	93.13	94.55	94.07	93.98	91.67	92.87	92.13	94.84	94.51	92.39	92.34	91.87	89.13	62.26	87.81
8	372	16	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	93.36	92.41	91.80	89.90	90.39	92.37	94.18	94.33	93.04	93.96	91.64	93.59	92.92	95.46	92.38	92.40	90.95	92.07	92.96	93.30	91.36	95.07	88.88	93.09	92.51
42	372	16	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	60.04	61.75	59.80	59.81	61.99	50.27	59.95	61.77	59.55	59.66	60.77	88.17	59.69	62.41	62.64	91.16	60.96	58.88	60.43	62.26	62.46	63.04	59.57	62.34	62.61
31	372	16	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	93.44	62.32	62.64	61.79	59.80	59.18	59.69	60.13	59.70	59.57	61.20	61.07	59.33	59.91	60.90	59.71	61.32	59.57	60.22	60.32	62.55	63.11	62.72	61.64	62.26
46	372	16	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	59.09	60.32	59.82	62.64	59.19	60.37	59.53	61.76	51.03	59.61	60.67	61.26	60.80	60.72	62.64	59.23	60.95	59.56	62.97	62.50	62.72	61.39	62.17	62.80	61.33
38	372	16	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	59.52	59.79	59.14	59.95	59.80	60.87	52.36	62.97	59.37	62.03	62.97	62.39	60.97	61.21	62.32	59.61	61.70	59.71	59.98	59.62	60.25	62.75	62.48	62.72	62.20
1	372	16	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	92.67	94.38	93.64	94.11	94.49	93.21	91.39	89.39	93.59	93.39	94.72	93.42	94.50	87.86	93.49	91.52	91.68	93.76	91.13	92.58	61.03	93.21	94.43	93.94	89.78
13	372	16	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	95.18	92.56	88.21	90.99	91.86	92.64	94.47	93.97	92.58	92.19	92.14	91.85	92.77	94.48	51.17	94.34	94.34	92.46	92.37	93.06	61.74	92.58	91.18	94.59	92.67
5	372	16	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	95.10	93.45	95.77	91.04	89.61	92.23	93.37	92.65	92.79	93.95	89.07	93.26	90.39	92.84	91.41	93.56	90.51	90.42	92.16	92.91	92.24	92.53	61.10	92.86	70.81
33	372	16	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	59.91	62.06	59.85	60.22	59.62	59.02	93.12	60.03	60.08	90.77	50.51	61.71	59.62	59.21	62.06	60.37	59.90	59.62	60.27	61.99	62.25	62.69	62.39	61.78	62.75
39	372	16	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	59.45	59.70	61.74	59.72	59.21	60.33	61.20	62.41	59.81	59.50	51.47	62.72	60.34	58.92	61.87	59.85	61.61	59.98	60.22	62.32	61.55	62.32	62.79	62.50	61.81
27	372	16	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	59.42	52.56	59.93	62.39	59.43	59.33	60.53	50.74	60.65	59.57	62.97	61.93	51.36	60.72	61.76	50.40	61.66	60.86	60.83	59.53	62.36	62.57	60.44	63.09	62.44
32	372	16	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	60.43	59.65	59.41	61.99	59.81	59.45	60.40	62.90	59.46	61.77	60.73	62.32	60.60	59.86	52.81	59.76	61.64	59.61	60.77	60.77	62.48	62.39	62.60	62.32	62.97
22	372	16	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	60.32	61.01	59.62	59.48	59.80	60.77	59.48	62.26	60.77	60.89	49.99	60.00	59.57	60.03	61.45	90.79	59.71	60.81	60.77	61.94	62.91	62.03	62.71	62.68	51.89
17	372	16	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	94.08	92.56	95.00	91.98	95.50	89.95	92.75	93.10	94.56	93.18	92.63	92.37	94.71	94.65	88.54	92.71	48.12	93.22	89.43	91.09	91.74	92.48	92.68	56.57	92.61
19	372	16	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	94.33	96.36	91.19	93.37	95.37	92.15	95.07	93.06	93.12	93.08	93.84	92.27	91.78	86.90	48.12	95.65	48.12	91.03	92.12	92.25	93.78	93.49	92.86	91.47	92.20
10	372	16	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	91.04	94.11	90.14	92.09	94.27	93.11	93.46	91.67	95.06	94.48	92.71	92.29	94.73	92.03	94.63	94.35	93.27	94.86	92.34	92.36	93.98	94.02	94.16	92.60	90.30
2	372	16	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	92.42	88.17	93.81	88.32	93.18	93.06	92.77	93.30	94.09	93.37	91.78	92.68	91.98	95.89	94.92	93.97	92.23	93.10	93.33	91.20	93.22	92.72	92.72	90.34	93.27
43	372	16	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	60.37	61.99	60.27	61.93	59.31	59.63	59.95	62.88	59.57	61.76	60.22	60.94	61.31	61.03	61.50	51.78	61.61	59.56	61.13	59.16	61.45	62.34	62.26	62.32	62.44
41	372	16	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	59.69	59.80	59.77	62.57	60.67	59.88	50.94	62.16	60.37	59.71	61.20	62.06	91.19	62.41	61.99	59.90	59.43	59.79	62.57	62.64	62.60	62.55	59.72	62.66	62.64
24	372	16	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	59.86	63.40	59.77	61.99	59.31	59.76	60.29	62.26	90.90	59.79	62.64	60.11	50.28	59.85	59.95	60.11	62.09	59.23	51.82	59.99	62.55	62.34	59.05	50.08	62.30
23	372	16	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	59.85	62.64	60.19	62.06	59.95	60.77	59.53	59.95	59.83	60.09	59.90	61.20	59.62	90.22	48.12	51.38	59.66	59.65	62.64	60.67	60.26	63.04	62.79	62.77	61.85
44	372	16	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	59.56	59.80	62.32	59.50	60.12	60.01	60.08	60.50	59.69	59.86	59.71	62.84	60.73	61.53	61.70	60.53	61.56	59.58	62.97	62.06	62.66	63.04	62.57	62.03	62.53
21	372	16	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	59.96	60.17	59.47	59.36	59.57	61.30	60.00	61.52	60.83	60.03	61.07	62.41	60.29	60.08	61.48	60.33	61.01	60.27	59.80	88.43	60.26	63.02	62.45	62.36	61.87

Size of the test data:  (47, 33)

Difference

diff_mw = dt_mw[['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
for i, cols in enumerate(zip(dt_mw.columns,test_data_mw1.columns)):   
   diff_mw[f'col_diff_{i}'] = (np.array(dt_mw[cols[0]]) - np.array(test_data_mw1[cols[1]]))*-1

# Function to count negatives in a row
def count_negatives(row):
    return sum(1 for value in row if isinstance(value, (int, float)) and value < 0)

diff_mw['negative_count'] = diff_mw.apply(count_negatives, axis=1)

diff_mw.sort_values(by =["negative_count"],inplace = True, ascending=False)


html_table = diff_mw.to_html(index=True)

# Wrap in a scrollable div
scrollable_table = f"""
<div style="height: 400px; width: 100%; overflow-x: auto; overflow-y: auto;">
    {html_table}
</div>
"""
# Display the scrollable table
display(HTML(scrollable_table))

# show(diff)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	col_diff_0	col_diff_1	col_diff_2	col_diff_3	col_diff_4	col_diff_5	col_diff_6	col_diff_7	col_diff_8	col_diff_9	col_diff_10	col_diff_11	col_diff_12	col_diff_13	col_diff_14	col_diff_15	col_diff_16	col_diff_17	col_diff_18	col_diff_19	col_diff_20	col_diff_21	col_diff_22	col_diff_23	col_diff_24	col_diff_25	col_diff_26	col_diff_27	col_diff_28	col_diff_29	col_diff_30	col_diff_31	col_diff_32	negative_count
1052	372	16	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.75	-6.31	-30.25	-29.74	-29.75	-32.25	-30.34	-35.66	-31.40	-31.82	-31.29	-30.66	-32.99	-31.35	-7.47	-33.10	-33.80	-31.43	-31.77	-33.43	5.22	1.08	-30.26	2.30	3.93	21
284	372	16	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.56	-5.67	-30.53	0.06	-0.70	-30.45	-30.16	3.14	-32.01	1.06	-32.26	-33.08	-29.79	-31.07	-1.03	-31.95	-6.68	-32.70	-31.48	-32.58	2.34	4.30	-29.49	3.97	2.77	18
531	372	16	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.06	-28.50	-28.94	-30.73	-30.11	-27.97	-30.24	2.25	-27.18	-29.38	-28.71	4.25	-30.07	-30.89	0.87	-30.58	1.87	-30.63	-32.64	-1.93	-29.12	5.24	4.32	4.98	3.48	17
1119	372	16	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-26.94	-0.62	-29.78	-31.23	-31.13	-29.61	-29.44	1.82	-28.59	-28.00	-39.75	-31.42	-32.10	-30.14	2.41	3.89	-32.46	-27.57	-29.86	4.30	6.46	4.29	4.87	5.82	-35.37	17
479	372	16	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.72	-1.06	-30.65	1.22	-29.54	-31.57	-32.15	2.01	-6.01	-32.07	-32.49	4.01	-30.89	1.69	-29.88	-32.51	-32.43	5.65	-32.94	-31.63	-27.80	7.45	-29.15	5.12	6.94	17
927	372	16	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.04	-34.53	-29.73	1.43	-29.79	-32.64	-25.30	-36.64	-29.70	-31.64	4.45	1.68	-35.68	-28.38	1.45	-38.32	1.96	-27.81	-29.56	-31.64	3.79	4.91	-27.61	5.39	2.80	16
1043	372	16	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.12	0.63	-5.29	1.17	-30.89	-31.69	-31.42	-29.96	-35.38	-31.11	-32.68	-32.41	-32.50	-31.76	0.46	-29.89	-32.26	0.69	-31.72	3.37	1.12	2.55	-31.88	3.14	2.26	16
848	372	16	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.01	0.51	-30.35	-0.32	0.75	-32.62	-30.82	-31.37	-31.50	-5.99	-32.02	3.14	-31.12	-32.16	1.37	-29.75	-0.09	-32.02	-30.90	-32.13	5.53	2.14	2.50	4.30	3.84	16
851	372	16	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.06	0.58	-0.07	0.88	-29.79	-32.72	-30.04	-31.31	-32.49	-31.52	-31.99	3.93	-32.21	-30.98	3.06	-33.69	3.72	-30.07	-31.52	-33.90	2.61	3.01	-30.37	4.86	4.51	16
348	372	16	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.80	2.96	-28.89	1.58	-30.41	-33.46	-29.62	1.84	3.00	-31.15	4.29	-33.07	-36.55	-31.57	-29.97	-31.31	2.61	-33.74	-37.34	-32.18	6.70	4.75	-33.41	-4.67	5.06	16
1436	372	16	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.25	0.75	-29.17	-29.57	-28.76	-30.00	-27.92	0.01	-0.06	-32.03	-31.21	-30.87	-27.66	-32.72	1.23	-29.67	0.26	-31.84	4.54	-31.41	4.19	5.01	-0.22	0.91	2.09	16
351	372	16	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.79	2.87	-27.45	0.24	-29.72	-28.37	-28.87	-29.21	-30.10	-29.81	-31.78	-27.83	-32.03	2.67	4.40	-35.30	-31.02	-30.77	5.48	-28.48	-28.36	6.21	3.96	3.42	1.82	16
924	372	16	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.75	-28.98	-1.69	-28.96	-29.99	-30.56	-24.59	3.51	-29.36	-31.21	-39.93	6.39	-29.02	-31.79	1.73	-31.34	1.97	-29.68	-30.44	3.15	5.32	6.32	5.63	6.35	2.86	15
1439	372	16	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.22	-29.78	-29.42	2.58	-31.93	-31.88	-31.98	1.63	-31.87	3.78	-28.87	-32.28	-31.82	2.69	1.27	-32.12	-30.94	-32.95	-37.20	-32.98	0.80	4.24	2.29	4.57	4.13	15
1503	372	16	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.91	-29.38	-29.01	-29.21	-29.52	-29.99	-33.52	4.43	-31.00	2.83	6.22	3.19	-27.30	-0.11	2.84	-32.57	2.19	-30.60	-33.49	-32.73	-29.27	4.63	4.92	5.99	3.81	15
912	372	16	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.75	-0.04	-27.83	-28.93	-29.85	-33.34	6.52	-29.34	-30.07	8.55	-5.50	2.50	-31.81	-32.73	2.28	-30.29	-31.02	-32.10	-31.90	3.47	6.60	6.11	4.96	2.76	5.92	15
1427	372	16	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.14	-0.21	-26.97	0.50	0.68	-30.01	-29.32	1.50	-31.57	0.40	-32.21	-32.44	-31.25	-31.95	-1.91	-31.23	1.55	-32.15	-33.49	3.28	3.32	3.48	-6.55	2.12	2.77	15
1107	372	16	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-26.96	-30.04	-29.31	0.92	-30.12	-32.77	-26.44	3.65	-31.00	2.35	-29.40	4.28	-29.75	-32.06	-35.96	-31.51	1.34	-30.81	-29.85	-29.86	5.42	4.92	4.93	2.65	5.86	15
287	372	16	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.00	-30.78	-29.14	1.34	-30.48	-32.23	-30.09	1.37	-32.31	-32.47	-30.66	3.20	-31.08	-32.48	-33.39	5.73	2.28	-31.29	3.26	-31.52	5.60	3.99	2.62	2.06	5.26	14
1491	372	16	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.18	1.15	2.84	1.53	-28.37	-32.54	-29.72	-30.77	-32.25	-29.85	-27.59	-28.73	-31.12	-31.01	1.60	-31.72	3.24	-31.10	-30.97	-30.60	4.49	8.25	4.20	2.69	3.03	14
336	372	16	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.27	0.94	-27.60	1.74	-31.16	-31.43	-29.71	3.27	-31.36	2.17	-31.84	-28.37	-26.49	2.20	2.29	-37.40	3.30	-31.87	-27.96	-32.80	2.66	4.03	3.07	2.67	4.74	13
1424	372	16	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.59	1.41	0.61	-31.06	1.41	-32.54	-30.52	1.39	-30.48	1.90	-32.85	4.16	-32.86	-31.17	1.08	-33.66	6.49	-32.32	-33.40	-32.32	-32.19	5.28	5.17	4.27	2.28	13
528	372	16	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.61	-31.12	3.06	-30.99	-28.28	-32.17	-29.82	-29.62	-31.01	-29.56	-32.76	6.54	-26.83	1.51	3.22	-29.84	2.55	-33.16	5.89	3.43	4.06	6.31	5.81	4.12	5.67	13
863	372	16	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.45	-30.36	-28.29	1.80	-28.84	-29.68	-30.39	1.60	-31.67	-32.16	6.44	-34.27	-29.20	1.17	1.88	-31.63	0.83	-31.67	1.68	-37.63	4.21	4.93	4.80	1.65	3.94	13
1500	372	16	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-31.45	-29.33	-28.58	2.47	-29.52	-31.27	-30.42	1.98	-36.33	-30.83	-29.46	-27.03	-27.09	1.20	3.28	-32.99	2.30	-31.12	6.46	3.50	6.82	2.35	4.89	6.63	2.05	13
339	372	16	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.72	-30.89	-26.88	2.17	-26.44	-32.30	-34.89	2.20	-29.78	-29.45	-28.34	3.54	6.68	4.48	1.61	-30.76	-32.28	-30.40	4.87	3.77	4.35	5.97	-30.70	6.05	5.00	13
1488	372	16	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.10	0.48	-28.37	-30.12	0.94	-37.05	-30.21	1.50	-32.36	-29.26	-28.84	6.83	-30.22	1.55	3.70	5.45	3.24	-33.83	-29.47	5.04	5.38	6.43	-30.29	5.04	6.73	12
607	372	16	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	24.78	20.65	5.99	-1.02	3.92	6.79	10.64	0.03	-0.09	0.86	4.03	24.28	2.04	2.03	-25.14	5.56	-28.32	6.93	2.29	13.03	0.45	2.40	2.09	2.78	-28.49	5
735	372	16	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	11.55	7.71	11.14	6.16	7.14	4.23	9.25	5.18	6.69	6.51	-1.24	5.83	0.74	7.18	2.94	8.06	-0.03	5.78	3.12	3.35	2.17	3.03	-27.24	2.49	-20.29	4
92	372	16	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.51	35.15	7.42	4.48	10.64	5.73	10.48	4.49	3.77	8.49	5.48	-0.11	3.81	9.39	-3.19	5.40	4.72	2.96	3.96	5.47	4.95	7.06	2.26	-36.77	-30.26	4
723	372	16	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.14	3.29	1.59	4.17	5.53	1.33	10.44	4.97	3.92	5.34	1.15	4.31	5.48	6.70	-37.88	7.68	7.16	4.36	3.55	-0.90	-29.38	3.34	4.01	38.61	2.65	3
1235	372	16	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	1.41	-1.72	9.66	-19.68	4.37	3.80	7.97	5.25	6.07	6.50	4.83	2.98	10.82	4.23	1.60	6.68	8.58	6.63	6.21	2.80	4.97	3.89	4.14	-28.69	1.62	3
1244	372	16	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	4.55	36.25	3.69	4.51	10.69	7.26	9.44	7.11	4.61	6.38	2.59	1.95	4.02	4.63	-0.42	6.18	2.13	3.85	5.96	-0.53	4.95	-0.98	2.12	2.51	4.27	3
1308	372	16	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.67	6.96	8.30	5.52	6.44	4.21	8.32	6.21	5.15	5.36	0.90	8.78	6.37	9.35	6.68	4.23	3.75	9.07	5.52	4.83	4.83	4.24	2.23	-32.28	-1.77	2
159	372	16	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.97	3.51	9.27	5.20	6.95	8.35	6.48	6.53	6.95	4.95	3.88	2.53	3.71	10.00	7.53	6.26	5.15	5.26	6.69	-1.24	4.05	2.33	5.78	-0.40	3.82	2
1232	372	16	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.45	2.37	9.31	4.96	4.70	1.80	8.41	0.84	9.58	1.91	6.62	0.18	8.93	3.89	-17.68	7.33	3.39	3.15	2.36	1.22	-6.91	2.57	5.98	2.75	1.93	2
659	372	16	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.46	6.82	8.80	7.50	11.27	5.67	9.62	6.20	8.57	4.10	5.99	3.93	2.37	2.57	-0.18	6.69	5.02	8.27	2.50	2.30	5.70	3.03	2.30	-31.19	5.65	2
668	372	16	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	5.26	5.96	9.07	6.64	10.02	6.27	10.23	-29.70	3.00	8.78	3.42	4.44	8.76	6.04	4.39	5.10	1.83	5.27	1.58	6.53	3.71	3.54	2.88	-30.27	5.01	2
1247	372	16	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	9.73	12.15	8.16	7.29	9.57	7.35	6.13	6.91	7.40	6.59	3.61	0.76	5.11	2.35	8.29	6.98	6.68	7.23	2.79	2.37	4.84	2.26	4.84	2.07	-2.40	1
144	372	16	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	10.41	7.77	8.87	5.40	10.29	1.53	6.92	5.09	7.38	5.57	4.18	3.72	7.85	6.78	0.24	37.46	4.40	5.07	2.02	2.11	3.08	4.22	4.29	-35.15	3.29	1
156	372	16	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	7.61	8.62	5.57	7.98	5.91	4.21	8.05	7.30	9.92	9.28	1.12	3.17	8.97	5.30	6.81	6.33	6.13	7.73	4.07	4.78	5.03	5.02	4.92	2.50	-0.24	1
720	372	16	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.63	7.96	10.02	7.08	10.17	6.69	4.86	4.03	5.01	8.43	7.26	5.00	7.51	1.55	5.85	5.07	1.33	5.60	0.61	1.49	-33.42	5.22	5.95	5.19	2.95	1
671	372	16	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	11.59	7.29	7.63	6.96	1.95	3.96	11.98	3.20	7.78	6.07	6.07	3.22	8.45	6.99	6.56	1.98	4.47	6.35	3.47	2.00	4.57	-30.86	2.50	4.24	3.90	1
83	372	16	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.61	5.80	3.77	5.80	9.52	6.37	7.84	7.07	5.87	7.97	3.43	7.00	3.76	6.28	4.24	7.22	2.59	1.60	2.73	4.56	-31.04	3.67	1.58	2.71	4.25	1
95	372	16	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	10.80	4.61	7.60	5.09	6.87	1.07	6.70	4.92	5.53	5.74	7.32	41.33	8.27	5.14	4.84	5.56	11.46	6.73	3.98	2.58	6.45	4.45	5.25	5.18	-22.33	1
147	372	16	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.20	10.75	7.35	3.59	8.88	7.57	8.65	5.18	8.22	4.34	6.30	0.59	4.61	0.13	4.40	9.22	4.40	1.96	2.56	1.97	4.92	4.59	4.70	1.06	3.67	0
1311	372	16	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	8.50	4.16	5.90	6.03	5.81	6.52	9.01	6.11	7.50	7.68	0.09	5.31	4.82	10.05	5.41	5.17	4.66	4.54	1.10	5.50	2.35	6.04	1.42	4.75	2.42	0

Model performance on valid data

Average & Std Deviation of the Significant rows:

	mean	std
mAP_valid_zero_6	89.582340	2.443592
mAP_valid_zero_18	88.547447	4.713353
mAP_valid_zero_13	88.389149	2.257730
mAP_valid_zero_16	88.259149	5.483069
mAP_valid_zero_9	87.331064	6.394417
mAP_valid_zero_7	87.227447	2.370679
mAP_valid_zero	86.595106	3.586833
mAP_valid_zero_11	86.467872	10.379424
mAP_valid_zero_19	85.155532	11.666241
mAP_valid_zero_5	84.694681	8.488761
mAP_valid_zero_20	83.973404	12.901407
mAP_valid_zero_3	83.660000	9.282097
mAP_valid_zero_10	83.540851	11.318962
mAP_valid_zero_14	83.232128	11.819109
mAP_valid_zero_12	79.447872	14.828596
mAP_valid_zero_4	77.781277	13.030367
mAP_valid_zero_23	77.762766	14.911936
mAP_valid_zero_8	77.308085	13.906354
mAP_valid_zero_2	76.469787	13.577989
mAP_valid_zero_17	75.532979	15.960938
mAP_valid_zero_21	74.431277	15.860655
mAP_valid_zero_15	73.067660	15.433913
mAP_valid_zero_26	72.012128	15.565705
mAP_valid_zero_22	70.991489	15.657550
mAP_valid_zero_25	70.979574	15.738005

Model performance on Test data

We then apply the hyperparameters to the test set and average the results.

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params = dt_valid.loc[dt_valid_sub_mw1.index].iloc[:, :6].reset_index()
optimal_params.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_1 = pd.merge(optimal_params, dt_test, how='left')
test_data_1_sub = test_data_1[list(test_data_1.columns[test_data_1.columns.str.startswith('mAP_test')])]
test_data_1['average_map'] = test_data_1_sub.apply(np.mean, axis=1)

test_data_1.sort_values(by=['average_map'], ascending= False,inplace=True)

	mean	std
mAP_test_zero_16	74.908404	17.186230
mAP_test_zero_7	74.803032	17.581260
mAP_test_zero_10	74.783830	16.537339
mAP_test_zero_12	74.731809	15.054400
mAP_test_zero	74.519149	16.663832
mAP_test_zero_20	74.303032	15.941213
mAP_test_zero_22	74.287766	14.747048
mAP_test_zero_9	74.282979	17.156531
mAP_test_zero_5	74.170532	16.436209
mAP_test_zero_13	74.121649	16.817738
mAP_test_zero_23	73.978191	15.824531
mAP_test_zero_3	73.966064	16.483464
mAP_test_zero_11	73.848936	16.349771
mAP_test_zero_14	73.845106	16.193388
mAP_test_zero_19	73.643298	16.140287
average_map	73.643030	14.836877
mAP_test_zero_8	73.638138	15.959009
mAP_test_zero_2	73.575851	15.724723
mAP_test_zero_18	73.463989	16.616144
mAP_test_zero_6	73.380266	16.421872
mAP_test_zero_26	73.023404	14.328836
mAP_test_zero_4	72.689202	15.402309
mAP_test_zero_17	72.668564	16.378359
mAP_test_zero_21	72.616170	14.914739
mAP_test_zero_25	71.057819	14.460644
mAP_test_zero_15	70.768564	15.483869

Summary using radar plot

res1_valid['id'] = res1_valid.index.to_series().apply(extract_number)
res1_test['id'] = res1_test.index.to_series().apply(extract_number)



res_comb = pd.concat([res1_valid,res1_test])
index_series = res_comb.index.to_series()

res_comb['type'] = np.where(
    index_series.str.contains('valid', case=False), 'valid',
    np.where(index_series.str.contains('test', case=False), 'test', 'unknown')
)
res_comb = res_comb.query("type !='unknown'")

res_comb =  res_comb.reset_index(drop=True)


res1_test = res1_test.sort_values(by=['id']).reset_index().query("index !='average_map'")

data_range1 = np.array(list(res1_valid['mean']) + list(res1_test['mean']))

categories = [str(i) for i in range(1,26)]

fig = go.Figure()

# Valid
fig.add_trace(go.Scatterpolar(
       r=list(res1_test['mean']),
      theta=categories,
      #fill='toself',
      name='Test'
))

# Test
fig.add_trace(go.Scatterpolar(
      r=list(res1_valid['mean']),
      theta=categories,
      #fill='toself',
      name='Valid'
))

# Customization of chart
fig.update_layout(
  title={
      'text': "T-Test",
        'xanchor': 'center',
      'yanchor': 'top'
  },
  polar=dict(
    radialaxis=dict(
      visible=True,
      range=[round(data_range1.min()), round(data_range1.max()) + 1]
    )),
  showlegend=True
)

fig.show()




##############


res2_valid['id'] = res2_valid.index.to_series().apply(extract_number)
res2_test['id'] = res2_test.index.to_series().apply(extract_number)


res_comb = pd.concat([res2_valid,res2_test])
index_series = res_comb.index.to_series()

res_comb['type'] = np.where(
    index_series.str.contains('valid', case=False), 'valid',
    np.where(index_series.str.contains('test', case=False), 'test', 'unknown')
)
res_comb = res_comb.query("type !='unknown'")

res_comb =  res_comb.reset_index(drop=True)


res2_test = res2_test.sort_values(by=['id']).reset_index().query("index !='average_map'")

data_range2 = np.array(list(res2_valid['mean']) + list(res2_test['mean']))

categories = [str(i) for i in range(1,26)]
fig = go.Figure()

# Valid
fig.add_trace(go.Scatterpolar(
       r=list(res2_test['mean']),
      theta=categories,
      #fill='toself',
      name='Test'
))

# Test
fig.add_trace(go.Scatterpolar(
      r=list(res2_valid['mean']),
      theta=categories,
      #fill='toself',
      name='Valid'
))

# Customization of chart
fig.update_layout(
  title={
      'text': "MU-Test",
         'xanchor': 'center',
      'yanchor': 'top'
  },
  polar=dict(
    radialaxis=dict(
      visible=True,
      range=[round(data_range2.min()), round(data_range2.max()) + 1]
    )),
  showlegend=True
)

fig.show()

best_valid_bit_size_16 = round(res1_valid['mean'][0])
id_best_valid = res1_valid['id'][0]
best_test_bit_size_16 = list(round(res1_test.query('id == @id_best_valid')['mean'],2))[0]

best_valid_bit_size_16_mw = round(res2_valid['mean'][0])
id_best_valid = res2_valid['id'][0]
best_test_bit_size_16_mw = list(round(res2_test.query('id == @id_best_valid')['mean'],2))[0]

Bit size 8

The results in this presentation are from two experimental designs:

Experiment 1 Results

The thresholding is based on fixed values between -1 and 1 on a step size of 0.1.

Validation Data mAP Results Preview:

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_abs_values	mAP_valid_abs_values_2	mAP_valid_abs_values_3	mAP_valid_abs_values_4	mAP_valid_abs_values_5	mAP_valid_abs_values_6	mAP_valid_abs_values_7	mAP_valid_abs_values_8	mAP_valid_abs_values_9	mAP_valid_abs_values_10	mAP_valid_abs_values_11	mAP_valid_abs_values_12	mAP_valid_abs_values_13	mAP_valid_abs_values_14	mAP_valid_abs_values_15	mAP_valid_abs_values_16	mAP_valid_abs_values_17	mAP_valid_abs_values_18	mAP_valid_abs_values_19	mAP_valid_abs_values_20	mAP_valid_abs_values_21	mAP_valid_abs_values_22	mAP_valid_abs_values_23	mAP_valid_abs_values_25	mAP_valid_abs_values_26
0	372	8	0.1	16	0.00100	24	0.000000e+00	0.000000e+00	65.21	63.17	63.44	63.04	52.78	59.88	63.32	60.91	60.67	61.82	60.93	64.43	58.38	58.88	62.46	58.99	61.72	63.82	60.18	63.30	67.91	60.79	60.67	62.62	65.74
3	372	8	0.1	16	0.00100	24	0.000000e+00	1.000000e-08	54.76	57.74	62.70	62.66	62.12	57.95	63.68	60.39	65.12	60.03	58.52	56.98	60.54	56.37	60.95	43.72	63.32	61.39	58.17	56.56	64.19	65.18	60.32	58.16	60.33
12	372	8	0.1	16	0.00100	24	1.000000e-08	0.000000e+00	56.36	43.72	69.57	64.69	62.58	62.35	57.81	62.85	66.27	58.15	43.72	62.19	61.92	64.66	64.56	61.61	64.14	60.49	62.02	57.84	60.34	43.72	65.59	64.62	63.26
15	372	8	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	65.64	65.06	64.95	65.96	64.71	62.16	78.06	57.81	63.14	65.11	63.03	62.97	65.27	64.14	64.50	61.53	63.07	69.06	59.89	59.13	64.92	67.46	65.81	66.75	69.14
64	372	8	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	55.28	43.72	87.33	63.49	62.86	56.05	88.30	58.71	63.75	65.12	58.90	60.04	89.95	55.99	60.43	60.67	60.64	59.96	56.73	59.34	66.49	68.13	61.54	61.95	58.24
67	372	8	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	86.26	62.62	86.84	61.63	60.48	43.72	57.24	61.21	87.21	64.21	88.46	59.04	61.32	58.84	59.32	90.48	61.27	88.08	92.21	56.29	60.43	60.39	59.65	59.73	59.58
76	372	8	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	64.71	60.70	62.10	87.85	61.14	91.88	88.30	59.57	43.72	87.16	59.27	92.18	84.02	59.93	61.20	88.62	60.33	59.00	90.56	55.89	59.57	60.94	56.45	60.44	61.52
79	372	8	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	88.57	62.53	86.74	61.84	61.35	92.60	88.93	61.47	89.55	62.20	92.17	59.70	84.94	62.20	43.72	60.71	59.85	61.03	59.30	58.30	64.25	62.34	59.50	62.20	58.68
128	372	8	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	62.03	61.34	60.30	61.36	63.42	56.16	60.56	60.65	88.97	60.91	59.58	57.89	89.23	60.05	59.88	92.08	59.57	60.55	56.38	57.96	58.51	58.41	57.68	59.23	57.82
131	372	8	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	88.93	61.65	86.30	61.21	62.11	59.05	88.19	51.70	56.80	55.28	58.69	57.99	90.41	60.40	58.68	60.03	59.32	89.00	59.30	88.92	57.61	59.95	57.11	59.23	58.37
140	372	8	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	89.47	61.30	61.12	62.20	87.85	90.42	89.08	61.95	60.60	82.15	89.54	57.66	91.40	60.55	61.03	88.85	92.48	60.05	58.38	58.68	57.06	62.07	90.09	59.18	57.38
143	372	8	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	60.70	64.71	61.81	88.72	59.45	59.74	88.60	59.60	60.91	62.62	91.43	59.14	90.95	60.33	60.39	84.69	57.49	89.97	43.72	58.65	56.85	59.50	58.97	58.05	59.58
192	372	8	0.1	16	0.00100	36	0.000000e+00	0.000000e+00	56.60	60.72	56.22	64.49	64.99	58.72	54.20	58.36	56.72	58.84	64.00	64.20	76.61	61.93	60.92	63.18	63.06	60.21	59.64	59.31	64.38	60.91	61.52	61.16	60.14
195	372	8	0.1	16	0.00100	36	0.000000e+00	1.000000e-08	86.22	56.44	59.08	61.09	57.09	54.34	87.14	58.01	59.11	58.59	56.23	54.75	88.02	57.93	57.87	90.35	50.34	59.48	57.45	55.60	62.30	61.45	58.33	58.53	56.75
204	372	8	0.1	16	0.00100	36	1.000000e-08	0.000000e+00	67.39	56.73	66.55	64.54	63.55	66.39	43.72	62.84	64.68	65.99	57.00	43.72	56.97	63.82	60.90	43.72	64.60	65.03	59.62	58.64	67.02	62.37	60.67	66.36	56.81
207	372	8	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	67.68	68.93	68.88	67.94	67.85	66.44	69.21	64.78	66.59	64.92	67.51	63.61	67.26	64.58	66.57	71.89	69.79	66.18	65.79	63.18	65.28	71.54	67.96	68.13	68.20
256	372	8	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	86.22	64.25	87.63	63.04	63.92	60.11	88.64	62.74	64.17	87.34	60.43	57.99	87.00	61.27	64.76	92.24	59.90	56.52	59.80	92.46	59.68	60.61	61.01	57.17	59.93
259	372	8	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	64.49	63.25	61.96	61.22	62.59	89.97	88.17	63.16	89.63	62.17	91.99	58.58	61.24	55.73	62.05	46.94	62.82	59.63	59.87	89.60	63.15	63.13	60.36	51.48	59.83
268	372	8	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	57.08	59.16	55.69	60.26	87.42	59.67	43.72	60.87	60.78	63.24	91.42	56.86	90.45	60.30	59.62	61.87	60.96	61.55	58.48	56.09	62.65	61.76	58.42	59.90	58.83
271	372	8	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	84.02	61.12	62.67	87.69	62.23	90.67	43.72	58.39	62.31	89.64	57.26	58.96	90.51	58.56	57.35	90.30	61.59	59.78	57.43	57.88	60.98	60.34	60.88	60.26	59.15
320	372	8	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	88.33	64.65	62.29	61.26	62.48	57.16	86.62	61.54	90.72	61.19	59.23	43.72	91.18	59.52	61.15	90.49	59.75	91.46	60.88	58.32	59.05	63.79	60.61	60.63	57.94
323	372	8	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	88.35	61.48	86.23	60.74	59.33	90.19	90.24	59.94	60.55	61.32	59.04	56.87	88.98	58.24	58.85	59.08	59.01	91.17	93.15	56.00	59.11	58.23	59.09	60.54	60.49
332	372	8	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	63.65	62.59	62.77	62.35	54.88	59.18	89.30	87.40	81.90	59.19	59.52	56.41	87.31	59.22	60.06	61.12	59.85	92.87	56.58	58.85	57.36	61.07	59.49	57.73	59.46
335	372	8	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	88.57	60.13	60.22	61.27	89.95	58.07	90.81	60.70	90.41	90.49	58.59	60.34	82.30	57.75	60.22	91.60	60.09	55.97	57.90	55.78	59.52	61.56	56.52	58.71	59.57
384	372	8	0.1	16	0.00100	48	0.000000e+00	0.000000e+00	43.72	63.23	59.14	60.74	61.19	59.62	87.09	60.39	61.71	58.80	59.24	60.09	61.28	51.72	58.53	61.14	63.53	57.64	50.50	58.81	59.70	61.13	62.95	58.74	62.90
387	372	8	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	68.45	66.44	57.87	62.97	57.16	43.72	79.22	78.65	63.73	65.90	62.13	60.09	89.08	60.59	63.28	54.04	63.36	64.43	57.30	61.77	63.84	61.54	64.79	67.58	60.06
396	372	8	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	67.38	63.45	66.94	67.16	65.95	59.19	56.12	64.03	66.74	64.89	57.94	65.81	59.34	86.95	61.57	65.74	64.13	66.61	61.11	66.23	65.37	65.37	62.85	67.57	65.24
399	372	8	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	83.10	62.62	61.91	53.71	60.73	82.66	54.56	53.37	58.61	58.43	56.55	55.61	87.32	60.34	60.79	59.95	58.87	89.28	56.05	55.64	59.89	59.42	59.44	61.00	61.00
448	372	8	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	55.60	62.13	62.42	59.34	61.50	58.40	85.28	60.54	61.05	89.74	90.96	58.29	89.42	63.18	61.85	89.23	61.40	61.25	58.81	58.62	57.76	61.72	61.68	62.71	59.37
451	372	8	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	88.14	62.84	61.63	60.46	88.38	58.93	89.46	88.86	61.80	61.29	59.60	58.61	89.41	59.21	61.06	55.82	59.76	89.98	58.68	87.52	63.16	61.20	60.36	58.98	60.35
460	372	8	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	62.67	62.25	59.80	63.73	60.95	58.72	87.94	61.41	61.33	61.80	56.90	88.06	88.81	60.84	60.82	56.16	61.57	59.62	60.02	58.75	62.16	58.56	62.15	59.11	59.52
463	372	8	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	89.13	62.47	61.20	62.12	62.08	58.95	87.86	62.03	89.96	62.83	61.02	55.65	43.72	60.36	61.93	92.22	63.77	60.02	58.25	56.71	61.97	57.70	88.11	61.05	60.48
512	372	8	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	88.05	60.63	62.90	61.37	62.77	90.72	62.94	59.83	62.66	62.24	58.92	57.94	43.72	59.72	60.89	81.90	60.98	59.81	59.28	60.36	59.35	60.99	61.19	58.64	58.74
515	372	8	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	64.30	62.68	56.39	61.62	61.85	91.93	89.97	61.43	91.16	60.66	58.14	55.66	86.05	59.59	61.17	85.13	63.22	60.63	59.68	59.13	59.96	60.77	57.69	57.76	59.63
524	372	8	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	85.94	62.43	61.67	61.85	61.70	88.70	88.42	60.99	90.72	61.07	91.47	56.56	61.31	58.08	60.61	55.24	61.79	91.22	56.56	55.89	58.00	59.59	59.73	58.32	59.35
527	372	8	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	88.01	62.31	62.86	62.19	62.80	59.34	88.41	61.60	91.55	85.37	61.19	59.33	90.96	60.55	60.55	88.35	59.50	60.20	59.18	58.00	88.72	58.71	59.77	59.87	56.81
576	372	8	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	67.58	66.82	66.14	66.51	74.73	61.83	57.37	67.78	69.95	68.43	62.89	66.38	70.33	65.18	65.88	63.67	68.64	65.64	61.94	62.41	67.30	66.56	69.13	66.34	68.04
579	372	8	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	58.62	62.09	65.54	66.28	61.68	60.81	65.21	61.38	59.34	64.62	59.60	59.97	62.47	59.40	64.30	64.62	64.82	61.93	60.60	59.91	66.04	64.38	64.45	67.52	63.01
588	372	8	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	85.53	62.74	64.39	64.03	86.68	60.35	65.91	62.88	63.17	63.80	60.97	61.54	62.83	61.20	65.91	60.72	60.69	63.47	60.69	58.51	63.84	64.44	62.72	64.53	62.64
591	372	8	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	87.82	61.15	63.85	65.91	66.36	43.72	58.43	63.10	65.90	65.99	63.33	59.86	63.16	59.63	61.53	63.75	56.38	65.10	61.91	60.13	65.01	43.72	43.72	62.59	61.89
640	372	8	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	86.78	62.72	64.46	62.61	88.36	59.86	60.86	60.01	63.54	90.09	91.73	57.20	91.82	60.32	60.06	60.89	60.79	63.65	57.56	59.02	63.41	62.58	59.24	63.81	63.14
643	372	8	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	63.48	65.16	64.33	64.38	64.33	89.10	88.51	61.65	43.72	63.83	58.95	60.21	63.52	62.82	63.10	88.23	60.38	84.69	60.37	59.06	62.90	64.29	60.74	60.24	62.88
652	372	8	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	86.67	64.49	61.42	62.08	60.72	90.32	86.21	59.43	88.09	61.24	90.18	59.40	90.28	59.57	62.97	60.40	61.65	58.75	59.69	58.14	59.88	62.49	58.55	60.71	62.26
655	372	8	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	60.99	61.79	63.98	60.98	63.08	43.72	63.70	58.97	62.03	62.99	59.46	59.76	87.64	60.93	61.28	60.69	63.07	64.57	56.54	58.96	62.21	62.36	62.14	60.79	60.87
704	372	8	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	65.25	62.53	62.98	88.91	63.72	59.18	66.74	61.76	63.52	64.78	91.15	60.24	91.00	61.57	61.56	91.71	61.03	90.47	57.54	91.96	65.10	58.98	60.12	58.59	65.50
707	372	8	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	86.35	62.85	61.44	61.59	87.12	59.82	87.94	60.66	56.85	61.07	59.00	60.06	90.95	59.56	61.09	92.23	60.74	61.24	58.62	58.07	60.56	58.11	59.27	57.87	59.97
716	372	8	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	88.62	62.81	63.61	43.72	88.97	90.42	87.91	60.78	85.04	89.48	60.52	57.91	89.44	57.22	59.08	86.23	59.34	60.60	57.05	92.21	87.43	58.46	58.25	60.01	59.03
719	372	8	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	63.03	87.05	56.42	62.66	88.59	58.44	90.22	59.83	61.53	60.80	55.15	59.01	90.54	58.68	63.21	61.24	59.99	91.11	60.77	89.13	59.43	58.44	61.41	60.70	62.49
768	372	8	0.1	16	0.00010	36	0.000000e+00	0.000000e+00	60.35	61.77	62.98	61.43	54.76	58.52	54.14	43.72	63.84	58.63	57.32	58.21	43.72	60.32	58.17	84.57	60.23	58.70	59.88	56.39	63.12	61.61	62.82	57.44	58.26
771	372	8	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	65.44	61.40	61.10	65.25	64.80	59.64	63.15	63.02	64.78	61.44	60.84	59.71	87.78	62.64	62.34	63.80	62.27	61.76	58.18	59.43	64.48	63.84	62.32	66.20	66.27
780	372	8	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	65.55	65.59	64.78	61.61	64.71	59.20	54.62	64.38	64.49	66.82	63.20	58.30	61.61	61.87	61.80	62.71	61.54	63.85	59.75	59.95	62.30	63.18	63.15	62.75	62.69
783	372	8	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	66.47	64.73	63.32	64.49	64.87	62.01	66.87	64.63	64.15	62.51	59.20	59.03	64.50	62.32	63.84	67.19	65.54	63.40	59.49	57.50	67.09	63.80	68.22	65.74	67.35
832	372	8	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	63.11	64.46	88.09	62.35	62.15	62.14	82.81	63.12	49.89	63.73	87.53	61.40	63.48	61.57	62.47	63.82	61.98	61.60	60.14	91.03	64.82	61.79	61.32	59.75	61.32
835	372	8	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	87.47	62.41	89.51	89.18	63.55	61.14	89.94	83.67	86.95	61.73	91.83	56.81	90.30	60.45	58.77	61.30	59.06	58.12	56.86	58.99	60.80	58.11	60.52	59.49	57.66
844	372	8	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	56.19	60.34	55.93	87.87	60.62	58.52	87.07	60.16	62.49	60.30	57.77	58.22	89.48	59.85	60.94	88.32	51.13	60.49	58.55	57.82	62.28	59.96	60.17	61.01	62.81
847	372	8	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	88.46	86.61	61.60	60.41	63.10	59.16	85.88	60.31	55.85	62.92	90.63	58.39	90.68	61.86	60.50	62.60	60.27	63.86	91.04	58.40	61.64	59.96	60.87	61.77	60.26
896	372	8	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	88.59	61.95	61.41	64.16	86.41	85.89	89.53	60.80	43.72	83.58	59.06	60.44	88.95	58.75	62.96	90.82	59.49	61.76	57.81	56.20	59.05	62.72	58.91	59.32	61.40
899	372	8	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	86.08	62.11	89.24	88.21	61.33	57.30	88.03	61.34	60.75	60.41	56.87	57.80	91.44	60.22	62.39	90.00	57.36	91.76	58.30	93.22	58.84	57.87	60.68	59.20	58.29
908	372	8	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	90.41	89.64	64.51	60.15	88.57	91.93	84.97	59.55	61.58	89.38	87.44	89.31	89.54	58.04	60.00	91.49	57.67	61.25	84.20	55.10	59.92	58.83	59.86	58.21	57.07
911	372	8	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	87.15	61.02	89.21	89.45	89.51	59.47	90.45	61.91	89.72	61.17	91.42	92.42	91.92	91.66	59.65	92.11	61.52	92.24	57.28	59.07	62.80	61.12	58.26	49.09	60.25
960	372	8	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	67.03	64.19	56.95	63.81	60.94	64.67	65.05	62.46	63.43	54.37	56.47	63.51	64.68	63.70	61.45	63.25	61.09	64.53	58.98	63.46	67.46	65.03	63.54	67.04	65.84
963	372	8	0.1	16	0.00010	48	0.000000e+00	1.000000e-08	60.85	63.50	63.62	61.26	62.15	58.41	65.24	60.98	62.46	61.48	59.95	56.54	62.70	60.54	62.86	63.08	61.40	64.59	56.59	55.71	63.80	61.48	56.99	62.79	59.08
972	372	8	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	65.10	67.66	68.14	66.10	64.86	62.40	65.70	65.89	65.97	64.24	60.81	60.81	54.60	64.30	65.57	64.52	65.70	67.77	60.74	61.45	64.57	64.24	60.94	64.34	62.28
975	372	8	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	63.33	62.80	61.79	64.95	65.84	59.80	55.52	60.82	64.90	61.96	60.72	58.82	58.24	60.20	61.61	64.52	62.89	62.43	58.23	57.86	64.64	62.46	62.56	62.69	67.06
1024	372	8	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	64.52	61.93	74.37	61.89	63.27	59.53	88.48	60.54	60.39	62.14	59.26	56.64	78.46	89.22	64.01	85.55	59.32	89.74	60.38	56.91	66.98	64.43	59.66	58.94	61.59
1027	372	8	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	64.88	61.51	86.76	60.04	65.18	90.99	87.70	60.12	61.58	43.72	90.08	91.36	61.08	85.45	58.94	56.47	61.12	59.12	90.94	56.10	61.38	60.35	61.98	55.09	61.57
1036	372	8	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	78.76	61.66	59.88	59.91	60.57	56.95	86.51	61.37	43.72	54.43	54.77	58.23	58.89	58.31	59.37	91.86	54.84	91.14	56.93	57.51	58.83	61.79	60.32	59.93	57.49
1039	372	8	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	55.62	63.66	87.85	61.11	87.49	59.58	62.96	58.74	61.71	62.46	58.36	57.84	62.81	62.08	58.59	88.74	60.57	87.11	91.85	57.35	83.91	62.08	59.05	60.13	60.43
1088	372	8	0.1	64	0.00010	48	0.000000e+00	0.000000e+00	89.47	61.29	88.70	61.99	62.29	59.45	56.28	60.52	90.47	43.72	59.18	59.07	90.70	91.71	43.72	62.26	43.72	61.05	59.01	58.09	58.11	60.42	59.54	61.45	60.28
1091	372	8	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	88.33	60.56	89.77	61.74	61.80	58.55	61.66	60.21	59.13	90.22	59.93	57.33	91.30	59.88	61.11	92.07	59.72	58.81	55.28	56.11	62.23	64.16	59.63	50.33	59.19
1100	372	8	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	87.45	64.21	62.42	62.60	63.56	91.34	89.45	62.28	60.89	60.11	59.25	59.21	92.45	60.52	61.51	90.03	60.77	60.42	92.24	86.88	59.79	60.05	57.43	59.66	49.26
1103	372	8	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	63.03	89.71	63.81	61.33	62.54	91.95	87.41	60.60	61.73	89.97	55.43	60.38	88.64	60.93	60.68	90.69	59.64	59.23	93.43	58.29	56.40	58.01	58.74	59.47	59.36
1152	372	8	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	68.37	65.62	62.78	62.22	66.23	59.08	55.79	65.66	67.73	66.64	61.94	59.19	65.92	69.33	63.05	58.05	64.53	65.59	48.51	61.12	65.82	66.16	67.21	65.33	65.58
1155	372	8	0.1	16	0.00001	24	0.000000e+00	1.000000e-08	84.07	59.17	60.58	61.11	61.11	85.43	55.73	59.37	54.57	88.88	61.75	57.39	59.28	58.89	57.25	45.64	58.46	56.28	57.90	80.36	63.41	66.53	60.01	58.10	62.74
1164	372	8	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	68.20	62.28	67.24	63.71	64.84	62.34	57.31	65.37	66.57	65.00	61.40	60.71	65.61	62.86	65.27	64.89	65.44	80.75	61.27	58.40	62.14	64.95	66.70	68.03	68.45
1167	372	8	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	64.74	63.62	56.51	62.24	61.92	62.08	65.69	61.39	63.12	64.24	60.56	61.79	87.10	60.02	65.89	63.95	61.63	62.28	59.09	59.26	64.43	62.57	64.12	63.12	62.98
1216	372	8	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	63.27	61.16	63.64	63.89	63.04	59.41	87.96	62.02	60.42	60.50	88.80	57.82	90.78	60.42	61.33	62.40	62.97	62.51	84.02	88.94	65.12	61.35	62.71	66.26	62.31
1219	372	8	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	88.66	88.44	62.31	61.96	60.94	91.28	87.20	63.64	89.36	62.82	86.98	90.70	56.04	60.84	61.22	84.67	59.35	61.39	59.15	59.83	63.82	61.67	61.13	63.62	59.53
1228	372	8	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	55.22	62.51	57.19	62.87	86.08	90.29	61.71	63.61	62.11	68.83	57.20	60.12	69.95	60.12	62.88	89.99	62.42	58.29	56.28	57.34	61.52	63.29	60.89	58.72	60.87
1231	372	8	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	88.83	62.63	62.88	61.70	61.02	91.52	62.38	60.62	62.78	62.57	91.21	60.65	92.54	83.03	62.55	91.64	60.60	61.60	60.32	57.10	60.35	62.67	62.53	61.87	60.43
1280	372	8	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	89.24	60.89	62.16	90.38	88.42	91.69	88.46	60.67	61.58	62.84	61.80	59.01	90.95	59.66	59.41	91.21	58.12	59.69	57.80	58.47	60.55	56.03	60.63	59.43	60.32
1283	372	8	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	55.38	62.48	60.66	60.26	86.25	57.11	90.22	60.94	83.33	59.57	56.40	87.88	91.58	58.22	61.16	60.05	59.09	57.29	58.02	57.66	57.78	55.95	56.51	58.19	58.06
1292	372	8	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	88.20	61.53	88.59	60.81	61.01	85.92	61.32	59.29	90.56	91.23	86.73	57.82	83.55	58.62	59.13	59.22	57.86	90.35	87.83	56.33	58.26	57.86	55.73	57.88	57.50
1295	372	8	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	62.34	63.77	61.30	60.57	89.71	91.03	83.00	60.50	88.65	61.48	59.83	58.93	91.43	57.74	60.53	90.94	60.79	62.66	58.86	58.52	61.72	63.82	50.42	59.06	56.91
1344	372	8	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	71.63	71.01	68.87	72.86	65.75	66.28	52.33	70.72	69.71	69.62	63.73	62.72	67.67	66.51	66.65	75.97	69.87	67.94	43.72	65.17	70.15	69.81	66.60	66.17	70.97
1347	372	8	0.1	16	0.00001	36	0.000000e+00	1.000000e-08	58.07	63.24	61.60	63.16	63.46	61.07	57.01	60.72	64.26	63.75	62.18	58.68	88.36	61.64	62.15	57.85	59.11	59.78	60.51	58.34	63.67	61.06	53.68	67.25	62.31
1356	372	8	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	66.02	67.08	63.26	64.44	63.59	62.18	86.68	61.56	66.04	64.95	61.64	60.61	64.28	63.33	65.01	63.63	64.21	64.56	61.17	60.41	69.03	67.75	63.79	66.70	63.47
1359	372	8	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	65.93	63.97	62.87	65.07	66.21	56.35	55.68	65.35	87.18	64.97	60.30	60.50	56.87	64.98	63.06	63.72	63.21	64.10	61.93	59.75	66.79	68.89	62.23	64.56	65.07
1408	372	8	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	63.59	61.88	87.69	62.64	58.90	88.74	56.93	61.04	61.49	59.27	59.34	57.65	88.67	56.66	59.35	61.06	59.49	58.53	58.58	55.78	62.25	63.75	61.14	59.60	60.00
1411	372	8	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	61.10	55.95	62.91	59.22	88.45	58.34	89.03	59.84	57.65	60.83	90.90	57.05	57.00	57.64	58.44	88.15	55.94	90.53	59.49	58.34	61.78	61.17	63.55	59.12	60.04
1420	372	8	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	58.00	63.55	65.35	64.01	63.21	90.13	66.64	65.16	62.63	61.24	59.39	61.20	89.55	90.74	62.02	60.47	62.25	62.40	59.10	59.11	61.70	61.66	61.13	68.24	60.07
1423	372	8	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	63.70	62.37	62.03	88.84	60.23	61.44	64.98	60.90	62.50	63.05	89.51	60.58	91.12	61.37	61.74	61.97	61.88	60.87	90.22	58.54	62.27	61.39	63.06	60.80	58.31
1472	372	8	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	88.72	61.06	86.23	61.50	62.09	60.60	86.75	61.45	90.22	62.58	60.17	56.69	92.51	60.05	59.89	59.00	59.55	91.71	58.66	90.48	64.72	55.75	59.63	61.24	59.41
1475	372	8	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	87.97	61.62	87.45	60.96	60.60	59.45	62.15	59.86	61.07	89.04	91.19	59.43	91.43	60.54	60.52	58.94	60.05	58.41	58.07	56.83	57.89	59.67	56.56	57.94	59.85
1484	372	8	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	90.45	62.04	61.79	61.93	62.60	57.29	50.46	61.15	63.20	90.47	91.63	60.30	89.72	59.84	58.61	60.09	58.93	59.32	89.80	57.92	62.67	60.97	59.45	58.26	59.37
1487	372	8	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	88.03	64.43	89.80	62.32	89.22	59.10	88.69	59.29	90.67	62.82	92.44	60.30	60.73	60.90	60.02	45.81	61.15	90.98	59.98	56.88	89.43	58.24	57.67	58.24	59.04
1536	372	8	0.1	16	0.00001	48	0.000000e+00	0.000000e+00	45.90	62.74	63.31	64.01	62.49	61.12	49.74	62.45	65.04	64.93	62.31	43.72	60.54	62.21	55.07	63.25	65.39	55.12	59.40	56.89	68.98	64.88	65.44	66.11	62.47
1539	372	8	0.1	16	0.00001	48	0.000000e+00	1.000000e-08	56.65	59.32	64.89	60.21	57.16	58.65	85.78	57.84	54.91	62.56	61.25	55.98	87.48	59.37	63.87	58.49	64.04	62.25	57.20	56.59	62.75	61.68	62.46	61.81	59.12
1548	372	8	0.1	16	0.00001	48	1.000000e-08	0.000000e+00	64.36	63.75	65.04	62.51	66.83	57.21	68.48	61.14	57.22	86.28	54.98	61.65	63.35	63.08	60.73	60.74	60.08	55.55	59.00	56.52	62.88	60.27	63.02	66.89	60.30
1551	372	8	0.1	16	0.00001	48	1.000000e-08	1.000000e-08	63.94	63.91	65.35	64.44	65.63	57.81	57.62	63.63	62.75	63.93	56.62	59.54	58.04	62.17	63.52	55.58	60.75	64.70	59.23	61.39	65.80	65.90	64.83	67.85	59.94

Test Data mAP Results Preview:

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_abs_values	mAP_test_abs_values_2	mAP_test_abs_values_3	mAP_test_abs_values_4	mAP_test_abs_values_5	mAP_test_abs_values_6	mAP_test_abs_values_7	mAP_test_abs_values_8	mAP_test_abs_values_9	mAP_test_abs_values_10	mAP_test_abs_values_11	mAP_test_abs_values_12	mAP_test_abs_values_13	mAP_test_abs_values_14	mAP_test_abs_values_15	mAP_test_abs_values_16	mAP_test_abs_values_17	mAP_test_abs_values_18	mAP_test_abs_values_19	mAP_test_abs_values_20	mAP_test_abs_values_21	mAP_test_abs_values_22	mAP_test_abs_values_23	mAP_test_abs_values_25	mAP_test_abs_values_26
0	372	8	0.1	16	0.00100	24	0.000000e+00	0.000000e+00	60.67	60.61	60.79	60.07	53.06	59.07	59.40	59.32	63.04	60.85	60.50	58.46	64.07	63.01	60.38	63.55	61.48	59.86	58.86	59.20	61.20	62.71	62.66	61.76	59.99
3	372	8	0.1	16	0.00100	24	0.000000e+00	1.000000e-08	49.82	58.05	59.20	59.52	59.48	60.40	61.02	60.01	60.38	60.51	58.43	58.40	61.05	49.93	59.66	48.12	59.87	61.90	60.19	57.58	61.27	61.74	63.03	61.42	62.59
12	372	8	0.1	16	0.00100	24	1.000000e-08	0.000000e+00	50.55	48.12	60.23	58.59	58.13	60.20	48.55	60.57	60.72	51.82	48.12	60.85	60.53	60.21	59.02	60.73	61.01	58.83	60.80	57.28	62.23	48.12	61.35	60.79	62.25
15	372	8	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	60.25	58.63	60.50	60.99	59.26	60.35	49.61	48.88	60.44	60.66	60.22	58.65	61.05	58.78	59.94	59.15	60.67	59.97	59.96	59.63	61.82	61.43	61.69	61.15	60.55
64	372	8	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	50.78	48.12	57.82	61.53	61.00	49.33	58.06	59.80	60.24	59.88	60.02	61.66	58.83	49.70	58.85	60.11	60.15	61.06	58.25	58.00	60.35	61.02	62.15	62.04	62.87
67	372	8	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	56.71	61.84	57.16	60.92	61.08	48.12	63.11	60.06	58.39	60.61	55.46	61.22	60.51	60.12	59.90	57.68	59.59	57.67	58.96	59.85	61.95	61.83	63.12	62.88	61.44
76	372	8	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	60.95	59.58	61.08	57.94	60.60	59.52	57.26	59.78	48.12	56.81	62.29	58.54	85.89	60.97	60.67	57.43	60.06	59.56	49.26	59.33	61.27	62.52	63.39	61.86	61.22
79	372	8	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	59.05	61.76	55.83	60.60	60.28	58.98	59.89	60.55	57.15	60.13	59.88	62.70	87.16	60.16	48.12	60.31	59.74	60.08	60.77	61.29	60.92	62.25	62.69	62.20	61.64
128	372	8	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	61.88	62.45	60.22	61.65	60.20	57.75	60.84	61.90	59.22	61.99	61.31	60.40	58.05	61.07	60.96	59.58	61.17	60.93	60.82	60.90	61.49	61.72	63.19	62.19	62.76
131	372	8	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	59.62	62.50	58.23	60.72	60.37	60.86	57.79	51.38	50.14	50.00	60.77	61.65	59.63	61.84	60.99	60.85	60.72	57.88	61.31	51.06	62.44	61.83	62.34	62.18	62.40
140	372	8	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	59.00	61.40	61.28	60.90	59.80	59.80	59.23	60.69	60.77	82.94	49.59	60.17	59.73	61.43	61.99	48.48	58.81	61.99	61.25	61.42	61.60	61.82	59.47	61.39	62.45
143	372	8	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	62.11	61.81	61.81	59.12	61.64	62.18	58.41	60.93	60.67	61.46	59.77	61.99	58.89	60.88	60.88	50.57	60.78	58.52	48.12	61.93	62.49	62.34	61.89	62.37	61.79
192	372	8	0.1	16	0.00100	36	0.000000e+00	0.000000e+00	48.93	60.53	49.93	60.43	60.34	58.38	50.99	61.58	49.42	61.16	57.98	57.89	51.34	59.12	59.50	59.69	60.00	59.88	59.57	57.74	60.99	63.38	62.04	60.92	59.58
195	372	8	0.1	16	0.00100	36	0.000000e+00	1.000000e-08	57.95	58.84	59.09	58.37	56.72	58.61	56.48	58.82	58.88	58.48	59.29	58.43	57.53	57.36	58.69	58.17	56.16	59.31	58.76	58.63	61.45	62.23	60.60	60.71	61.63
204	372	8	0.1	16	0.00100	36	1.000000e-08	0.000000e+00	60.32	48.70	60.91	59.44	61.07	60.99	48.12	60.10	59.17	60.47	50.65	48.12	49.65	57.20	61.02	48.12	60.61	59.66	59.73	58.80	60.99	64.15	62.29	61.29	60.83
207	372	8	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	60.28	60.82	59.19	60.77	60.90	60.16	59.94	61.04	60.33	60.56	60.48	60.03	60.91	61.07	60.33	57.46	58.94	60.43	60.18	59.77	61.13	59.88	60.71	61.06	60.09
256	372	8	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	57.17	61.06	58.92	61.99	60.47	60.84	58.17	62.50	60.99	58.02	62.78	59.25	57.89	60.58	60.95	59.02	60.57	50.47	60.48	59.35	61.95	62.78	62.90	63.48	62.10
259	372	8	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	61.16	60.37	61.35	60.33	61.22	58.91	57.47	61.67	57.69	59.51	57.43	61.43	61.07	49.19	61.04	75.37	59.66	61.18	60.09	59.57	61.78	62.14	61.94	45.10	61.96
268	372	8	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	49.76	60.67	50.25	59.93	59.04	59.75	48.12	61.40	61.53	60.03	57.61	59.74	58.83	60.48	60.79	59.37	61.01	60.72	59.99	58.39	61.25	60.38	61.74	62.05	60.74
271	372	8	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	56.15	60.80	61.73	58.90	58.42	57.76	48.12	59.86	61.02	58.63	59.31	61.15	57.62	60.68	60.34	57.59	60.75	62.30	59.20	61.25	62.19	62.17	61.60	61.72	60.55
320	372	8	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	58.01	61.29	61.62	61.59	60.77	60.06	56.64	61.70	58.40	61.11	61.19	48.12	58.59	60.72	60.79	58.74	60.57	58.51	61.65	61.36	61.92	61.68	62.40	62.50	62.12
323	372	8	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	57.83	61.81	59.17	61.87	60.13	56.22	58.72	62.00	60.51	60.54	61.59	61.65	59.34	60.08	61.59	59.51	60.88	57.91	58.61	60.58	61.90	62.40	61.64	61.73	61.84
332	372	8	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	61.32	61.59	61.28	61.59	50.26	62.32	58.89	59.82	92.17	60.55	60.77	59.80	50.64	61.94	60.77	60.50	61.93	58.51	60.77	61.98	62.43	62.48	61.53	62.87	62.68
335	372	8	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	57.96	60.93	61.17	61.87	59.53	60.65	58.88	62.13	58.08	58.25	61.59	61.25	85.74	61.29	60.59	59.02	60.02	50.06	60.74	59.62	61.72	61.51	63.02	61.99	61.76
384	372	8	0.1	16	0.00100	48	0.000000e+00	0.000000e+00	48.12	58.35	60.24	60.16	58.92	59.13	57.33	59.19	59.99	59.18	59.21	59.24	60.57	53.04	59.25	60.43	60.45	62.22	54.59	58.98	62.24	61.99	62.01	63.69	61.68
387	372	8	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	60.28	60.99	49.50	59.37	50.77	48.12	49.53	34.94	60.22	60.52	59.73	59.64	57.88	59.10	60.57	62.90	59.64	59.94	58.51	59.68	61.56	60.80	60.91	60.38	62.57
396	372	8	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	59.09	60.24	61.49	60.70	59.90	48.56	50.43	59.64	58.99	59.19	58.53	59.02	46.16	59.23	59.32	58.50	60.31	60.21	58.66	58.28	61.41	62.22	61.58	60.97	61.89
399	372	8	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	55.86	58.70	58.98	49.98	61.71	48.76	48.08	51.65	57.46	58.75	59.64	59.71	57.37	60.54	60.53	59.20	57.84	56.23	50.08	58.70	60.87	60.70	61.82	61.25	61.23
448	372	8	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	49.74	60.73	59.18	59.49	60.95	58.26	57.41	61.39	59.76	58.22	57.78	60.47	57.69	60.39	60.02	57.40	60.77	61.26	60.48	59.54	62.63	60.89	61.77	62.06	61.07
451	372	8	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	57.49	60.49	60.37	61.25	57.13	60.99	59.19	59.88	60.64	60.85	60.94	60.77	57.44	60.33	60.83	50.25	60.55	57.60	59.32	51.34	61.52	59.81	61.80	61.51	62.18
460	372	8	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	61.24	60.14	66.53	62.06	61.15	60.17	59.28	60.46	60.77	61.35	60.09	60.33	58.07	60.76	60.17	49.26	59.84	58.92	61.99	60.66	61.79	60.20	62.55	62.67	61.81
463	372	8	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	57.99	60.64	60.62	62.36	60.89	58.77	57.33	60.08	58.88	60.71	61.62	58.59	48.12	59.63	60.71	59.56	61.02	60.08	61.37	59.38	62.26	62.76	60.03	62.20	61.50
512	372	8	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	57.85	58.74	61.53	61.64	61.40	58.30	61.26	61.53	60.07	60.74	61.59	61.18	48.12	60.75	60.93	85.97	60.97	61.11	61.31	61.41	63.03	61.69	61.39	62.29	61.43
515	372	8	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	61.23	61.66	49.59	61.66	61.16	59.90	57.99	61.70	58.21	60.99	62.70	50.25	50.54	60.47	62.00	50.65	60.64	61.39	61.37	59.62	61.63	61.85	62.67	63.04	61.99
524	372	8	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	56.61	60.61	61.38	61.71	61.18	56.17	58.35	61.65	59.67	61.29	58.86	59.92	61.45	61.22	61.36	76.02	60.98	58.99	59.67	59.90	62.24	62.15	61.83	61.67	60.93
527	372	8	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	59.57	61.65	61.75	61.97	61.93	60.21	57.26	61.17	59.84	90.67	61.59	61.84	59.60	62.08	61.88	50.58	60.67	62.08	61.56	60.65	60.46	62.85	62.18	62.57	62.59
576	372	8	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	60.45	61.30	59.49	60.16	61.74	60.65	47.99	59.83	59.84	59.66	58.99	59.42	59.25	65.80	60.63	57.96	59.14	58.87	59.85	59.29	61.48	60.29	60.60	61.10	60.83
579	372	8	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	48.54	60.64	59.58	61.28	59.33	60.09	59.38	60.90	49.46	61.36	60.48	59.51	61.29	59.35	59.93	58.57	60.40	59.43	60.17	59.24	60.95	61.78	60.88	60.48	61.94
588	372	8	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	56.41	61.73	59.97	59.28	58.14	59.05	60.52	59.74	58.57	59.96	59.29	61.46	59.41	59.37	60.99	60.52	59.27	59.80	60.03	58.47	60.92	62.67	62.42	62.53	62.51
591	372	8	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	59.49	59.41	59.55	60.24	59.07	48.12	48.39	58.98	60.07	59.18	61.13	58.90	61.37	59.84	58.91	61.47	50.34	59.79	60.61	60.54	61.86	48.12	48.12	62.06	62.20
640	372	8	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	57.99	60.21	59.58	61.00	57.73	60.33	58.21	59.95	61.74	58.42	59.64	58.78	58.51	60.82	60.46	59.90	60.54	60.27	59.71	60.40	61.68	62.03	62.78	61.74	61.32
643	372	8	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	61.36	60.61	61.13	61.09	61.63	59.73	58.49	61.39	48.12	61.11	60.42	60.58	60.42	60.41	59.95	57.77	60.18	82.74	60.06	60.70	60.98	61.74	61.66	62.07	60.83
652	372	8	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	56.37	61.66	60.05	61.79	59.96	57.62	56.24	61.40	58.62	61.16	59.15	61.47	58.97	60.51	61.25	60.77	60.48	60.63	61.58	59.21	62.44	61.45	61.79	61.67	61.68
655	372	8	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	60.40	62.42	61.22	62.00	61.47	48.12	61.49	60.20	61.09	61.03	61.52	61.65	57.44	60.59	61.80	60.65	60.10	59.48	60.04	61.15	61.62	61.27	60.53	62.37	61.60
704	372	8	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	61.16	61.87	61.27	59.81	61.81	60.11	59.64	61.76	60.67	61.39	60.33	62.39	59.28	60.90	61.75	59.43	61.49	59.45	62.97	59.04	60.33	62.55	62.20	62.42	61.84
707	372	8	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	60.22	60.66	61.89	61.29	57.85	61.99	59.09	62.22	48.88	62.32	60.77	61.77	59.21	61.36	61.48	59.67	60.88	61.75	61.47	61.81	62.45	62.79	62.81	62.54	62.19
716	372	8	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	59.31	61.99	61.92	48.12	59.52	60.33	57.82	61.82	49.18	58.63	61.99	61.99	59.42	61.82	60.75	51.35	61.10	60.51	61.65	58.84	59.87	62.26	61.49	62.49	61.77
719	372	8	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	61.05	59.74	50.01	61.03	59.41	60.41	58.25	60.55	59.85	61.37	49.75	61.99	59.20	60.18	61.50	60.74	60.85	59.52	61.64	88.98	62.50	62.36	61.50	62.01	61.18
768	372	8	0.1	16	0.00010	36	0.000000e+00	0.000000e+00	58.39	60.70	59.18	58.28	50.71	59.58	50.84	48.12	58.12	61.11	58.42	58.52	48.12	61.08	59.95	59.86	66.08	62.36	57.38	57.74	61.19	62.29	61.48	62.89	62.60
771	372	8	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	60.37	60.53	59.30	59.82	59.35	59.11	58.86	59.02	59.90	58.47	60.42	58.41	57.62	59.61	60.41	59.59	59.15	58.18	58.40	58.78	61.00	62.25	62.75	61.69	61.80
780	372	8	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	60.00	61.22	59.12	59.67	58.37	58.81	50.37	61.32	59.24	60.53	60.17	59.41	60.04	59.35	60.91	59.38	59.46	59.86	59.17	59.99	62.31	60.74	61.62	61.28	62.22
783	372	8	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	60.56	59.71	59.33	60.97	60.61	59.52	60.07	60.49	58.70	60.13	59.75	59.84	59.18	60.14	59.75	60.20	58.81	59.11	58.85	58.62	60.71	61.40	60.08	61.70	60.09
832	372	8	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	60.48	61.67	58.67	59.36	61.23	60.98	58.96	60.32	53.08	61.16	47.75	61.53	60.64	60.57	60.71	59.77	60.35	60.99	62.28	58.05	61.06	62.44	62.17	61.79	62.48
835	372	8	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	58.08	61.37	58.96	58.38	60.80	61.85	58.29	89.01	56.62	61.50	58.78	59.20	58.43	60.99	60.38	59.68	60.16	59.31	60.31	60.53	61.24	62.19	62.16	61.55	60.72
844	372	8	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	49.44	60.31	50.47	59.82	60.71	59.42	47.21	61.57	61.39	60.81	58.51	59.61	57.42	61.12	60.53	56.01	57.28	61.29	61.66	60.56	61.87	60.21	62.32	61.67	61.06
847	372	8	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	58.93	57.38	60.72	58.88	60.38	60.76	56.88	61.05	48.85	61.35	59.16	60.53	58.76	60.47	60.43	59.71	59.80	61.07	58.03	60.54	62.28	60.86	60.80	61.32	60.97
896	372	8	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	58.90	61.42	61.01	61.89	59.72	90.60	57.93	61.16	48.12	49.42	61.53	60.11	57.97	61.67	61.07	59.29	62.18	61.67	61.25	59.62	61.79	62.34	62.00	62.89	62.17
899	372	8	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	55.59	61.42	59.07	59.73	61.53	59.14	59.76	62.11	62.00	60.96	59.71	61.04	59.04	61.46	62.16	57.13	61.68	58.35	61.65	59.00	60.92	62.45	62.07	62.63	62.41
908	372	8	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	57.80	59.56	61.33	60.44	58.48	59.95	51.41	62.00	61.36	59.18	51.55	51.81	60.01	61.70	62.25	59.72	61.00	61.46	87.31	59.06	61.43	61.60	61.15	62.61	62.49
911	372	8	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	59.59	62.00	59.07	59.52	59.22	61.90	59.49	60.91	59.71	62.32	60.22	59.95	59.59	58.21	62.28	59.05	60.03	59.22	62.90	60.65	61.57	62.42	62.09	48.62	62.01
960	372	8	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	60.48	58.84	49.00	61.14	60.04	60.17	61.04	58.86	60.07	50.20	50.66	60.31	58.74	60.01	59.48	59.73	62.43	58.82	59.54	59.23	61.64	62.09	62.32	67.02	61.17
963	372	8	0.1	16	0.00010	48	0.000000e+00	1.000000e-08	59.83	61.15	59.44	60.53	58.33	58.73	59.53	58.36	57.54	60.46	59.76	60.96	60.62	59.45	59.28	59.53	58.27	59.73	59.78	59.01	60.52	58.70	60.99	62.26	61.56
972	372	8	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	59.24	59.36	60.97	61.00	60.42	59.84	61.00	64.43	60.39	59.34	59.26	59.12	52.63	59.28	61.40	58.31	59.07	58.26	60.52	60.05	61.14	61.89	62.95	60.19	61.50
975	372	8	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	59.51	59.23	58.49	59.84	60.77	58.76	49.29	59.67	58.20	58.83	60.22	58.62	48.01	59.47	59.33	60.43	62.39	59.73	58.66	59.10	61.94	61.43	61.83	62.20	60.55
1024	372	8	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	61.27	61.25	52.36	59.48	61.44	62.71	57.80	59.08	60.15	60.77	60.37	50.57	47.44	58.54	57.87	50.54	58.64	56.67	60.82	58.25	61.46	60.99	62.49	62.74	61.97
1027	372	8	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	60.53	59.99	58.64	60.24	60.74	59.67	58.80	59.26	58.73	48.12	50.01	59.15	59.59	47.94	60.59	49.59	60.45	60.70	58.70	59.60	60.63	62.15	60.19	62.50	61.08
1036	372	8	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	87.56	60.87	60.22	60.96	58.87	58.74	50.02	59.07	48.12	63.17	51.59	60.22	59.64	61.28	60.48	57.85	50.78	56.76	60.44	60.06	61.60	62.23	61.82	62.20	60.54
1039	372	8	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	50.29	61.89	57.17	60.76	57.43	61.04	60.65	60.24	60.21	60.19	59.18	60.79	59.52	60.47	61.51	57.43	60.94	85.19	58.88	60.60	85.64	62.23	62.47	61.80	61.80
1088	372	8	0.1	64	0.00010	48	0.000000e+00	0.000000e+00	58.72	60.17	59.06	61.71	61.64	61.61	49.90	61.99	58.61	48.12	61.99	61.75	58.92	60.02	48.12	61.46	48.12	61.75	61.99	60.81	62.48	61.92	62.45	62.43	61.15
1091	372	8	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	58.45	59.52	58.63	60.77	62.06	60.59	61.26	61.05	61.14	58.85	62.39	61.31	58.87	60.85	61.91	58.48	61.09	61.40	50.00	59.57	60.04	61.91	61.03	50.48	62.19
1100	372	8	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	59.69	61.55	61.21	61.31	60.72	59.48	59.22	61.70	61.51	61.44	61.87	61.58	59.52	61.48	61.70	57.64	60.82	62.09	59.37	52.38	62.90	62.33	62.28	62.56	46.25
1103	372	8	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	61.88	59.48	61.29	60.75	61.99	58.62	59.27	61.15	61.02	59.12	59.17	61.99	60.19	60.59	60.40	59.37	61.36	60.98	59.95	61.87	62.48	62.69	62.56	62.08	62.25
1152	372	8	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	60.37	59.61	62.79	61.30	60.04	58.65	48.61	60.23	60.99	61.33	59.69	59.64	61.55	58.70	61.14	64.98	60.01	59.73	45.95	58.82	61.66	60.99	61.35	61.42	61.67
1155	372	8	0.1	16	0.00001	24	0.000000e+00	1.000000e-08	55.41	60.20	59.56	60.05	60.82	56.21	50.23	58.47	50.55	57.07	59.69	58.91	61.46	59.56	59.04	44.10	59.55	49.20	59.70	82.65	61.67	60.51	62.71	62.61	59.08
1164	372	8	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	60.29	59.06	60.26	59.11	58.96	59.30	49.40	59.95	57.95	58.74	60.44	59.07	61.23	61.30	59.04	59.70	60.46	59.21	60.62	57.95	62.32	59.21	62.06	60.46	61.03
1167	372	8	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	59.77	60.61	50.31	61.56	60.59	59.52	61.64	59.37	61.15	60.78	60.79	59.54	58.83	59.18	61.36	59.91	60.02	59.35	59.66	59.65	60.04	61.29	61.47	61.44	61.43
1216	372	8	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	60.29	58.88	58.27	59.47	60.96	58.90	58.23	59.86	58.78	60.37	51.23	59.47	58.42	58.87	59.18	61.58	60.81	62.28	87.45	56.55	61.40	61.39	61.63	61.38	61.64
1219	372	8	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	58.28	59.36	61.75	60.33	60.39	59.30	58.88	61.50	58.12	61.65	50.92	59.48	50.25	60.22	61.29	52.21	60.78	59.76	59.47	61.65	61.61	62.72	61.79	61.42	61.71
1228	372	8	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	50.68	60.37	49.94	61.59	55.85	59.35	59.72	62.02	58.91	75.73	59.57	60.83	45.60	60.14	60.46	58.86	59.30	60.66	50.32	60.08	61.04	62.18	62.88	61.84	62.40
1231	372	8	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	58.67	60.27	60.46	61.03	59.32	60.02	61.50	61.64	59.92	61.77	59.33	61.59	59.18	86.26	59.51	59.70	60.43	61.45	61.53	59.63	63.13	61.59	62.18	61.63	62.10
1280	372	8	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	59.53	61.03	61.48	60.14	59.80	59.61	59.30	62.09	60.34	61.14	61.59	61.99	59.28	61.18	60.86	58.80	60.32	61.80	60.11	61.12	63.47	62.84	62.55	62.18	61.89
1283	372	8	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	50.30	60.97	60.71	61.29	58.25	59.37	59.32	61.70	87.38	61.66	60.97	50.82	59.55	61.47	61.93	61.64	60.74	60.09	62.17	61.94	62.08	62.60	62.08	61.61	60.91
1292	372	8	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	59.39	61.99	58.51	61.33	61.43	51.96	59.62	61.93	58.49	58.87	86.78	61.70	49.68	60.50	60.58	61.39	59.97	59.52	50.60	61.87	62.20	62.13	62.58	62.33	61.14
1295	372	8	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	61.10	60.69	61.93	61.81	59.71	58.52	90.79	61.70	58.97	62.08	60.65	61.49	59.23	62.39	60.97	59.27	60.04	60.14	62.64	61.68	61.42	61.55	49.99	61.84	62.18
1344	372	8	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	58.91	58.94	61.11	58.72	61.35	56.96	50.01	59.52	59.68	60.07	58.32	57.76	60.52	60.60	61.13	61.70	59.03	61.34	48.12	57.87	59.88	60.80	61.63	61.83	62.05
1347	372	8	0.1	16	0.00001	36	0.000000e+00	1.000000e-08	47.82	61.17	59.82	59.57	60.34	59.40	49.53	60.21	60.47	59.85	61.06	57.91	57.64	60.18	60.69	51.20	59.20	62.34	60.38	50.44	62.57	62.78	43.86	59.36	62.42
1356	372	8	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	59.76	60.13	60.05	60.35	60.05	59.96	57.98	59.20	60.40	60.34	62.12	59.63	59.23	59.66	60.04	60.79	59.02	58.91	60.55	59.14	60.72	60.68	61.87	60.69	61.55
1359	372	8	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	59.03	61.01	60.33	60.32	61.74	49.79	49.14	59.19	56.83	59.25	62.16	60.12	49.87	59.80	59.64	60.08	59.62	59.50	59.18	59.86	59.54	59.35	61.98	61.48	61.16
1408	372	8	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	61.11	60.40	57.03	60.92	60.10	56.97	49.51	60.34	60.39	61.21	59.43	59.66	58.03	50.16	59.89	60.37	61.39	62.21	60.96	50.31	61.75	61.65	62.56	61.71	62.00
1411	372	8	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	59.34	49.49	59.98	59.43	59.49	60.58	58.35	61.24	58.61	59.85	56.87	60.20	50.35	59.94	58.70	57.54	50.11	57.48	61.77	60.65	61.76	61.25	60.71	60.97	61.43
1420	372	8	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	50.11	61.27	58.98	61.63	60.02	58.25	59.78	61.15	60.09	59.62	62.13	61.93	58.32	59.24	60.30	59.93	60.67	60.68	61.03	60.19	61.77	62.31	62.36	60.26	62.08
1423	372	8	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	61.92	61.03	61.37	57.93	60.93	61.45	60.20	60.77	61.42	61.28	55.55	61.59	58.63	61.33	60.50	60.24	60.29	60.38	57.35	61.53	60.75	62.25	62.11	61.45	61.00
1472	372	8	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	58.88	62.32	56.32	60.97	61.41	60.92	88.18	61.11	57.61	60.48	62.32	59.93	58.31	62.18	61.58	60.71	60.72	58.94	60.46	50.61	60.25	49.51	62.36	62.52	61.00
1475	372	8	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	59.15	61.47	59.52	61.06	61.29	61.68	61.33	61.11	61.59	58.82	59.95	62.26	59.33	62.15	61.94	61.23	61.26	59.76	61.16	61.59	62.42	61.83	62.55	61.26	62.11
1484	372	8	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	59.32	59.92	61.75	60.41	61.02	49.75	51.94	61.93	61.51	59.39	57.41	61.65	59.15	61.21	60.51	58.17	60.23	60.93	52.20	60.44	61.80	62.36	61.79	61.53	62.39
1487	372	8	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	59.74	61.52	59.26	62.32	59.41	62.08	58.77	61.93	59.10	61.42	59.79	62.15	62.16	61.21	61.42	65.45	60.75	59.48	61.53	59.45	60.03	60.47	61.95	61.82	62.28
1536	372	8	0.1	16	0.00001	48	0.000000e+00	0.000000e+00	65.84	60.24	60.10	59.36	60.05	59.22	52.83	61.08	60.87	58.17	60.09	48.12	61.69	59.97	51.91	59.40	59.82	49.84	59.64	58.52	60.46	61.24	61.34	61.46	60.41
1539	372	8	0.1	16	0.00001	48	0.000000e+00	1.000000e-08	50.63	58.35	60.87	58.59	49.01	60.10	56.52	58.89	50.75	59.44	58.07	59.70	57.61	60.23	60.41	57.36	59.25	59.84	59.32	58.67	61.77	60.74	62.07	62.18	61.88
1548	372	8	0.1	16	0.00001	48	1.000000e-08	0.000000e+00	59.36	59.74	59.79	59.97	60.43	59.43	60.07	59.30	49.35	56.92	51.45	61.13	59.62	58.60	62.08	61.55	57.92	50.00	59.68	50.13	63.10	62.31	62.37	60.38	62.83
1551	372	8	0.1	16	0.00001	48	1.000000e-08	1.000000e-08	60.89	60.25	58.80	60.59	60.03	57.87	49.64	59.64	59.02	59.01	49.99	60.37	49.19	61.63	60.34	51.26	60.23	59.44	59.77	60.04	61.33	61.10	61.75	60.55	62.46

Model performance on validation data (T-test)

Statistical Significance:

Using the global mean maximum row as the reference, we perform the right-tailed t-test to identify significant hyperparameters.

pvalues = []
pvalues_real = []

for _,index in enumerate(dt_valid_sub.index):
  _, p_value = stats.ttest_ind(max_row[:-1], dt_valid_sub.loc[index][:-1], alternative='greater')
  pvalues_real.append(p_value)
  pvalues.append((p_value >= 0.05)*1)

  

dt_valid_sub['pvalues'] = pvalues_real
dt_valid_sub['sig'] = pvalues


dt_valid_sub1 = dt_valid_sub.query('sig == 1')


max_indexx = dt_valid_sub1.iloc[:, :-3].mean().idxmax()

#print(sum(pvalues),max_indexx)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_abs_values	mAP_valid_abs_values_2	mAP_valid_abs_values_3	mAP_valid_abs_values_4	mAP_valid_abs_values_5	mAP_valid_abs_values_6	mAP_valid_abs_values_7	mAP_valid_abs_values_8	mAP_valid_abs_values_9	mAP_valid_abs_values_10	mAP_valid_abs_values_11	mAP_valid_abs_values_12	mAP_valid_abs_values_13	mAP_valid_abs_values_14	mAP_valid_abs_values_15	mAP_valid_abs_values_16	mAP_valid_abs_values_17	mAP_valid_abs_values_18	mAP_valid_abs_values_19	mAP_valid_abs_values_20	mAP_valid_abs_values_21	mAP_valid_abs_values_22	mAP_valid_abs_values_23	mAP_valid_abs_values_25	mAP_valid_abs_values_26
1219	372	8	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	88.66	88.44	62.31	61.96	60.94	91.28	87.20	63.64	89.36	62.82	86.98	90.70	56.04	60.84	61.22	84.67	59.35	61.39	59.15	59.83	63.82	61.67	61.13	63.62	59.53
1231	372	8	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	88.83	62.63	62.88	61.70	61.02	91.52	62.38	60.62	62.78	62.57	91.21	60.65	92.54	83.03	62.55	91.64	60.60	61.60	60.32	57.10	60.35	62.67	62.53	61.87	60.43
835	372	8	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	87.47	62.41	89.51	89.18	63.55	61.14	89.94	83.67	86.95	61.73	91.83	56.81	90.30	60.45	58.77	61.30	59.06	58.12	56.86	58.99	60.80	58.11	60.52	59.49	57.66
1027	372	8	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	64.88	61.51	86.76	60.04	65.18	90.99	87.70	60.12	61.58	43.72	90.08	91.36	61.08	85.45	58.94	56.47	61.12	59.12	90.94	56.10	61.38	60.35	61.98	55.09	61.57
67	372	8	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	86.26	62.62	86.84	61.63	60.48	43.72	57.24	61.21	87.21	64.21	88.46	59.04	61.32	58.84	59.32	90.48	61.27	88.08	92.21	56.29	60.43	60.39	59.65	59.73	59.58
76	372	8	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	64.71	60.70	62.10	87.85	61.14	91.88	88.30	59.57	43.72	87.16	59.27	92.18	84.02	59.93	61.20	88.62	60.33	59.00	90.56	55.89	59.57	60.94	56.45	60.44	61.52
79	372	8	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	88.57	62.53	86.74	61.84	61.35	92.60	88.93	61.47	89.55	62.20	92.17	59.70	84.94	62.20	43.72	60.71	59.85	61.03	59.30	58.30	64.25	62.34	59.50	62.20	58.68
256	372	8	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	86.22	64.25	87.63	63.04	63.92	60.11	88.64	62.74	64.17	87.34	60.43	57.99	87.00	61.27	64.76	92.24	59.90	56.52	59.80	92.46	59.68	60.61	61.01	57.17	59.93
451	372	8	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	88.14	62.84	61.63	60.46	88.38	58.93	89.46	88.86	61.80	61.29	59.60	58.61	89.41	59.21	61.06	55.82	59.76	89.98	58.68	87.52	63.16	61.20	60.36	58.98	60.35
1280	372	8	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	89.24	60.89	62.16	90.38	88.42	91.69	88.46	60.67	61.58	62.84	61.80	59.01	90.95	59.66	59.41	91.21	58.12	59.69	57.80	58.47	60.55	56.03	60.63	59.43	60.32
1292	372	8	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	88.20	61.53	88.59	60.81	61.01	85.92	61.32	59.29	90.56	91.23	86.73	57.82	83.55	58.62	59.13	59.22	57.86	90.35	87.83	56.33	58.26	57.86	55.73	57.88	57.50
1472	372	8	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	88.72	61.06	86.23	61.50	62.09	60.60	86.75	61.45	90.22	62.58	60.17	56.69	92.51	60.05	59.89	59.00	59.55	91.71	58.66	90.48	64.72	55.75	59.63	61.24	59.41
1487	372	8	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	88.03	64.43	89.80	62.32	89.22	59.10	88.69	59.29	90.67	62.82	92.44	60.30	60.73	60.90	60.02	45.81	61.15	90.98	59.98	56.88	89.43	58.24	57.67	58.24	59.04
704	372	8	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	65.25	62.53	62.98	88.91	63.72	59.18	66.74	61.76	63.52	64.78	91.15	60.24	91.00	61.57	61.56	91.71	61.03	90.47	57.54	91.96	65.10	58.98	60.12	58.59	65.50
716	372	8	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	88.62	62.81	63.61	43.72	88.97	90.42	87.91	60.78	85.04	89.48	60.52	57.91	89.44	57.22	59.08	86.23	59.34	60.60	57.05	92.21	87.43	58.46	58.25	60.01	59.03
896	372	8	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	88.59	61.95	61.41	64.16	86.41	85.89	89.53	60.80	43.72	83.58	59.06	60.44	88.95	58.75	62.96	90.82	59.49	61.76	57.81	56.20	59.05	62.72	58.91	59.32	61.40
899	372	8	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	86.08	62.11	89.24	88.21	61.33	57.30	88.03	61.34	60.75	60.41	56.87	57.80	91.44	60.22	62.39	90.00	57.36	91.76	58.30	93.22	58.84	57.87	60.68	59.20	58.29
908	372	8	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	90.41	89.64	64.51	60.15	88.57	91.93	84.97	59.55	61.58	89.38	87.44	89.31	89.54	58.04	60.00	91.49	57.67	61.25	84.20	55.10	59.92	58.83	59.86	58.21	57.07
911	372	8	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	87.15	61.02	89.21	89.45	89.51	59.47	90.45	61.91	89.72	61.17	91.42	92.42	91.92	91.66	59.65	92.11	61.52	92.24	57.28	59.07	62.80	61.12	58.26	49.09	60.25
1100	372	8	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	87.45	64.21	62.42	62.60	63.56	91.34	89.45	62.28	60.89	60.11	59.25	59.21	92.45	60.52	61.51	90.03	60.77	60.42	92.24	86.88	59.79	60.05	57.43	59.66	49.26
1103	372	8	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	63.03	89.71	63.81	61.33	62.54	91.95	87.41	60.60	61.73	89.97	55.43	60.38	88.64	60.93	60.68	90.69	59.64	59.23	93.43	58.29	56.40	58.01	58.74	59.47	59.36
140	372	8	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	89.47	61.30	61.12	62.20	87.85	90.42	89.08	61.95	60.60	82.15	89.54	57.66	91.40	60.55	61.03	88.85	92.48	60.05	58.38	58.68	57.06	62.07	90.09	59.18	57.38
323	372	8	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	88.35	61.48	86.23	60.74	59.33	90.19	90.24	59.94	60.55	61.32	59.04	56.87	88.98	58.24	58.85	59.08	59.01	91.17	93.15	56.00	59.11	58.23	59.09	60.54	60.49
335	372	8	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	88.57	60.13	60.22	61.27	89.95	58.07	90.81	60.70	90.41	90.49	58.59	60.34	82.30	57.75	60.22	91.60	60.09	55.97	57.90	55.78	59.52	61.56	56.52	58.71	59.57
527	372	8	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	88.01	62.31	62.86	62.19	62.80	59.34	88.41	61.60	91.55	85.37	61.19	59.33	90.96	60.55	60.55	88.35	59.50	60.20	59.18	58.00	88.72	58.71	59.77	59.87	56.81

Size of the All data:  (100, 28)
Size of the Sig data:  (25, 28)

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params1 = dt_valid.loc[dt_valid_sub1.index][['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
optimal_params = optimal_params1.reset_index()
optimal_params.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_1 = pd.merge(optimal_params, dt_test, on=['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg'])
test_data_1_sub = test_data_1[list(test_data_1.columns[test_data_1.columns.str.startswith('mAP_test')])]

Equivalent test data

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_abs_values	mAP_test_abs_values_2	mAP_test_abs_values_3	mAP_test_abs_values_4	mAP_test_abs_values_5	mAP_test_abs_values_6	mAP_test_abs_values_7	mAP_test_abs_values_8	mAP_test_abs_values_9	mAP_test_abs_values_10	mAP_test_abs_values_11	mAP_test_abs_values_12	mAP_test_abs_values_13	mAP_test_abs_values_14	mAP_test_abs_values_15	mAP_test_abs_values_16	mAP_test_abs_values_17	mAP_test_abs_values_18	mAP_test_abs_values_19	mAP_test_abs_values_20	mAP_test_abs_values_21	mAP_test_abs_values_22	mAP_test_abs_values_23	mAP_test_abs_values_25	mAP_test_abs_values_26
4	372	8	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	58.28	59.36	61.75	60.33	60.39	59.30	58.88	61.50	58.12	61.65	50.92	59.48	50.25	60.22	61.29	52.21	60.78	59.76	59.47	61.65	61.61	62.72	61.79	61.42	61.71
17	372	8	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	58.67	60.27	60.46	61.03	59.32	60.02	61.50	61.64	59.92	61.77	59.33	61.59	59.18	86.26	59.51	59.70	60.43	61.45	61.53	59.63	63.13	61.59	62.18	61.63	62.10
9	372	8	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	58.08	61.37	58.96	58.38	60.80	61.85	58.29	89.01	56.62	61.50	58.78	59.20	58.43	60.99	60.38	59.68	60.16	59.31	60.31	60.53	61.24	62.19	62.16	61.55	60.72
21	372	8	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	60.53	59.99	58.64	60.24	60.74	59.67	58.80	59.26	58.73	48.12	50.01	59.15	59.59	47.94	60.59	49.59	60.45	60.70	58.70	59.60	60.63	62.15	60.19	62.50	61.08
23	372	8	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	56.71	61.84	57.16	60.92	61.08	48.12	63.11	60.06	58.39	60.61	55.46	61.22	60.51	60.12	59.90	57.68	59.59	57.67	58.96	59.85	61.95	61.83	63.12	62.88	61.44
11	372	8	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	60.95	59.58	61.08	57.94	60.60	59.52	57.26	59.78	48.12	56.81	62.29	58.54	85.89	60.97	60.67	57.43	60.06	59.56	49.26	59.33	61.27	62.52	63.39	61.86	61.22
19	372	8	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	59.05	61.76	55.83	60.60	60.28	58.98	59.89	60.55	57.15	60.13	59.88	62.70	87.16	60.16	48.12	60.31	59.74	60.08	60.77	61.29	60.92	62.25	62.69	62.20	61.64
10	372	8	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	57.17	61.06	58.92	61.99	60.47	60.84	58.17	62.50	60.99	58.02	62.78	59.25	57.89	60.58	60.95	59.02	60.57	50.47	60.48	59.35	61.95	62.78	62.90	63.48	62.10
18	372	8	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	57.49	60.49	60.37	61.25	57.13	60.99	59.19	59.88	60.64	60.85	60.94	60.77	57.44	60.33	60.83	50.25	60.55	57.60	59.32	51.34	61.52	59.81	61.80	61.51	62.18
15	372	8	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	59.53	61.03	61.48	60.14	59.80	59.61	59.30	62.09	60.34	61.14	61.59	61.99	59.28	61.18	60.86	58.80	60.32	61.80	60.11	61.12	63.47	62.84	62.55	62.18	61.89
5	372	8	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	59.39	61.99	58.51	61.33	61.43	51.96	59.62	61.93	58.49	58.87	86.78	61.70	49.68	60.50	60.58	61.39	59.97	59.52	50.60	61.87	62.20	62.13	62.58	62.33	61.14
14	372	8	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	58.88	62.32	56.32	60.97	61.41	60.92	88.18	61.11	57.61	60.48	62.32	59.93	58.31	62.18	61.58	60.71	60.72	58.94	60.46	50.61	60.25	49.51	62.36	62.52	61.00
7	372	8	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	59.74	61.52	59.26	62.32	59.41	62.08	58.77	61.93	59.10	61.42	59.79	62.15	62.16	61.21	61.42	65.45	60.75	59.48	61.53	59.45	60.03	60.47	61.95	61.82	62.28
8	372	8	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	61.16	61.87	61.27	59.81	61.81	60.11	59.64	61.76	60.67	61.39	60.33	62.39	59.28	60.90	61.75	59.43	61.49	59.45	62.97	59.04	60.33	62.55	62.20	62.42	61.84
3	372	8	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	59.31	61.99	61.92	48.12	59.52	60.33	57.82	61.82	49.18	58.63	61.99	61.99	59.42	61.82	60.75	51.35	61.10	60.51	61.65	58.84	59.87	62.26	61.49	62.49	61.77
24	372	8	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	58.90	61.42	61.01	61.89	59.72	90.60	57.93	61.16	48.12	49.42	61.53	60.11	57.97	61.67	61.07	59.29	62.18	61.67	61.25	59.62	61.79	62.34	62.00	62.89	62.17
6	372	8	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	55.59	61.42	59.07	59.73	61.53	59.14	59.76	62.11	62.00	60.96	59.71	61.04	59.04	61.46	62.16	57.13	61.68	58.35	61.65	59.00	60.92	62.45	62.07	62.63	62.41
1	372	8	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	57.80	59.56	61.33	60.44	58.48	59.95	51.41	62.00	61.36	59.18	51.55	51.81	60.01	61.70	62.25	59.72	61.00	61.46	87.31	59.06	61.43	61.60	61.15	62.61	62.49
0	372	8	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	59.59	62.00	59.07	59.52	59.22	61.90	59.49	60.91	59.71	62.32	60.22	59.95	59.59	58.21	62.28	59.05	60.03	59.22	62.90	60.65	61.57	62.42	62.09	48.62	62.01
12	372	8	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	59.69	61.55	61.21	61.31	60.72	59.48	59.22	61.70	61.51	61.44	61.87	61.58	59.52	61.48	61.70	57.64	60.82	62.09	59.37	52.38	62.90	62.33	62.28	62.56	46.25
13	372	8	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	61.88	59.48	61.29	60.75	61.99	58.62	59.27	61.15	61.02	59.12	59.17	61.99	60.19	60.59	60.40	59.37	61.36	60.98	59.95	61.87	62.48	62.69	62.56	62.08	62.25
2	372	8	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	59.00	61.40	61.28	60.90	59.80	59.80	59.23	60.69	60.77	82.94	49.59	60.17	59.73	61.43	61.99	48.48	58.81	61.99	61.25	61.42	61.60	61.82	59.47	61.39	62.45
20	372	8	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	57.83	61.81	59.17	61.87	60.13	56.22	58.72	62.00	60.51	60.54	61.59	61.65	59.34	60.08	61.59	59.51	60.88	57.91	58.61	60.58	61.90	62.40	61.64	61.73	61.84
22	372	8	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	57.96	60.93	61.17	61.87	59.53	60.65	58.88	62.13	58.08	58.25	61.59	61.25	85.74	61.29	60.59	59.02	60.02	50.06	60.74	59.62	61.72	61.51	63.02	61.99	61.76
16	372	8	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	59.57	61.65	61.75	61.97	61.93	60.21	57.26	61.17	59.84	90.67	61.59	61.84	59.60	62.08	61.88	50.58	60.67	62.08	61.56	60.65	60.46	62.85	62.18	62.57	62.59

Size of the test data:  (25, 33)

Difference

diff = dt[['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
for i, cols in enumerate(zip(dt.columns,test_data_1.columns)):   
   diff[f'col_diff_{i}'] = (np.array(dt[cols[0]]) - np.array(test_data_1[cols[1]]))*-1

# Function to count negatives in a row
def count_negatives(row):
    return sum(1 for value in row if isinstance(value, (int, float)) and value < 0)

diff['negative_count'] = diff.apply(count_negatives, axis=1)
diff.sort_values(by =["negative_count"],inplace = True, ascending=False)

html_table = diff.to_html(index=True)

# Wrap in a scrollable div
scrollable_table = f"""
<div style="height: 400px; width: 100%; overflow-x: auto; overflow-y: auto;">
    {html_table}
</div>
"""
# Display the scrollable table
display(HTML(scrollable_table))

# 
# show(diff)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	col_diff_0	col_diff_1	col_diff_2	col_diff_3	col_diff_4	col_diff_5	col_diff_6	col_diff_7	col_diff_8	col_diff_9	col_diff_10	col_diff_11	col_diff_12	col_diff_13	col_diff_14	col_diff_15	col_diff_16	col_diff_17	col_diff_18	col_diff_19	col_diff_20	col_diff_21	col_diff_22	col_diff_23	col_diff_24	col_diff_25	col_diff_26	col_diff_27	col_diff_28	col_diff_29	col_diff_30	col_diff_31	col_diff_32	negative_count
1219	372	8	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.38	-29.08	-0.56	-1.63	-0.55	-31.98	-28.32	-2.14	-31.24	-1.17	-36.06	-31.22	-5.79	-0.62	0.07	-32.46	1.43	-1.63	0.32	1.82	-2.21	1.05	0.66	-2.20	2.18	18
1027	372	8	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-4.35	-1.52	-28.12	0.20	-4.44	-31.32	-28.90	-0.86	-2.85	4.40	-40.07	-32.21	-1.49	-37.51	1.65	-6.88	-0.67	1.58	-32.24	3.50	-0.75	1.80	-1.79	7.41	-0.49	18
1231	372	8	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.16	-2.36	-2.42	-0.67	-1.70	-31.50	-0.88	1.02	-2.86	-0.80	-31.88	0.94	-33.36	3.23	-3.04	-31.94	-0.17	-0.15	1.21	2.53	2.78	-1.08	-0.35	-0.24	1.67	18
79	372	8	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.52	-0.77	-30.91	-1.24	-1.07	-33.62	-29.04	-0.92	-32.40	-2.07	-32.29	3.00	2.22	-2.04	4.40	-0.40	-0.11	-0.95	1.47	2.99	-3.33	-0.09	3.19	-0.00	2.96	17
704	372	8	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-4.09	-0.66	-1.71	-29.10	-1.91	0.93	-7.10	-0.00	-2.85	-3.39	-30.82	2.15	-31.72	-0.67	0.19	-32.28	0.46	-31.02	5.43	-32.92	-4.77	3.57	2.08	3.83	-3.66	16
911	372	8	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.56	0.98	-30.14	-29.93	-30.29	2.43	-30.96	-1.00	-30.01	1.15	-31.20	-32.47	-32.33	-33.45	2.63	-33.06	-1.49	-33.02	5.62	1.58	-1.23	1.30	3.83	-0.47	1.76	16
256	372	8	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.05	-3.19	-28.71	-1.05	-3.45	0.73	-30.47	-0.24	-3.18	-29.32	2.35	1.26	-29.11	-0.69	-3.81	-33.22	0.67	-6.05	0.68	-33.11	2.27	2.17	1.89	6.31	2.17	15
451	372	8	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.65	-2.35	-1.26	0.79	-31.25	2.06	-30.27	-28.98	-1.16	-0.44	1.34	2.16	-31.97	1.12	-0.23	-5.57	0.79	-32.38	0.64	-36.18	-1.64	-1.39	1.44	2.53	1.83	15
76	372	8	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-3.76	-1.12	-1.02	-29.91	-0.54	-32.36	-31.04	0.21	4.40	-30.35	3.02	-33.64	1.87	1.04	-0.53	-31.19	-0.27	0.56	-41.30	3.44	1.70	1.58	6.94	1.42	-0.30	14
1103	372	8	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-1.15	-30.23	-2.52	-0.58	-0.55	-33.33	-28.14	0.55	-0.71	-30.85	3.74	1.61	-28.45	-0.34	-0.28	-31.32	1.72	1.75	-33.48	3.58	6.08	4.68	3.82	2.61	2.89	14
1100	372	8	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.76	-2.66	-1.21	-1.29	-2.84	-31.86	-30.23	-0.58	0.62	1.33	2.62	2.37	-32.93	0.96	0.19	-32.39	0.05	1.67	-32.87	-34.50	3.11	2.28	4.85	2.90	-3.01	13
67	372	8	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.55	-0.78	-29.68	-0.71	0.60	4.40	5.87	-1.15	-28.82	-3.60	-33.00	2.18	-0.81	1.28	0.58	-32.80	-1.68	-30.41	-33.25	3.56	1.52	1.44	3.47	3.15	1.86	13
716	372	8	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.31	-0.82	-1.69	4.40	-29.45	-30.09	-30.09	1.04	-35.86	-30.85	1.47	4.08	-30.02	4.60	1.67	-34.88	1.76	-0.09	4.60	-33.37	-27.56	3.80	3.24	2.48	2.74	13
896	372	8	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.69	-0.53	-0.40	-2.27	-26.69	4.71	-31.60	0.36	4.40	-34.16	2.47	-0.33	-30.98	2.92	-1.89	-31.53	2.69	-0.09	3.44	3.42	2.74	-0.38	3.09	3.57	0.77	13
908	372	8	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-32.61	-30.08	-3.18	0.29	-30.09	-31.98	-33.56	2.45	-0.22	-30.20	-35.89	-37.50	-29.53	3.66	2.25	-31.77	3.33	0.21	3.11	3.96	1.51	2.77	1.29	4.40	5.42	12
1472	372	8	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.84	1.26	-29.91	-0.53	-0.68	0.32	1.43	-0.34	-32.61	-2.10	2.15	3.24	-34.20	2.13	1.69	1.71	1.17	-32.77	1.80	-39.87	-4.47	-6.24	2.73	1.28	1.59	12
140	372	8	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.47	0.10	0.16	-1.30	-28.05	-30.62	-29.85	-1.26	0.17	0.79	-39.95	2.51	-31.67	0.88	0.96	-40.37	-33.67	1.94	2.87	2.74	4.54	-0.25	-30.62	2.21	5.07	12
1487	372	8	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.29	-2.91	-30.54	-0.00	-29.81	2.98	-29.92	2.64	-31.57	-1.40	-32.65	1.85	1.43	0.31	1.40	19.64	-0.40	-31.50	1.55	2.57	-29.40	2.23	4.28	3.58	3.24	11
1280	372	8	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.71	0.14	-0.68	-30.24	-28.62	-32.08	-29.16	1.42	-1.24	-1.70	-0.21	2.98	-31.67	1.52	1.45	-32.41	2.20	2.11	2.31	2.65	2.92	6.81	1.92	2.75	1.57	11
835	372	8	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.39	-1.04	-30.55	-30.80	-2.75	0.71	-31.65	5.34	-30.33	-0.23	-33.05	2.39	-31.87	0.54	1.61	-1.62	1.10	1.19	3.45	1.54	0.44	4.08	1.64	2.06	3.06	11
527	372	8	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.44	-0.66	-1.11	-0.22	-0.87	0.87	-31.15	-0.43	-31.71	5.30	0.40	2.51	-31.36	1.53	1.33	-37.77	1.17	1.88	2.38	2.65	-28.26	4.14	2.41	2.70	5.78	11
899	372	8	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.49	-0.69	-30.17	-28.48	0.20	1.84	-28.27	0.77	1.25	0.55	2.84	3.24	-32.40	1.24	-0.23	-32.87	4.32	-33.41	3.35	-34.22	2.08	4.58	1.39	3.43	4.12	10
1292	372	8	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.81	0.46	-30.08	0.52	0.42	-33.96	-1.70	2.64	-32.07	-32.36	0.05	3.88	-33.87	1.88	1.45	2.17	2.11	-30.83	-37.23	5.54	3.94	4.27	6.85	4.45	3.64	9
323	372	8	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.52	0.33	-27.06	1.13	0.80	-33.97	-31.52	2.06	-0.04	-0.78	2.55	4.78	-29.64	1.84	2.74	0.43	1.87	-33.26	-34.54	4.58	2.79	4.17	2.55	1.19	1.35	9
335	372	8	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.61	0.80	0.95	0.60	-30.42	2.58	-31.93	1.43	-32.33	-32.24	3.00	0.91	3.44	3.54	0.37	-32.58	-0.07	-5.91	2.84	3.84	2.20	-0.05	6.50	3.28	2.19	9

Model performance on valid data

Average & Std Deviation of the Significant rows:

	mean	std
mAP_valid_abs_values_13	84.4564	11.376413
mAP_valid_abs_values	84.3564	8.921018
mAP_valid_abs_values_7	84.3216	10.150079
mAP_valid_abs_values_16	79.5260	15.968719
mAP_valid_abs_values_6	76.1992	16.910606
mAP_valid_abs_values_11	74.0268	15.727979
mAP_valid_abs_values_3	73.6320	12.956814
mAP_valid_abs_values_9	72.4084	16.013372
mAP_valid_abs_values_5	71.6496	13.036881
mAP_valid_abs_values_10	71.6288	13.972789
mAP_valid_abs_values_18	70.9076	15.265083
mAP_valid_abs_values_19	68.7420	15.399282
mAP_valid_abs_values_4	67.5056	12.867583
mAP_valid_abs_values_20	66.6412	15.394766
mAP_valid_abs_values_2	65.4016	9.059835
mAP_valid_abs_values_12	65.3108	13.271108
mAP_valid_abs_values_21	64.0056	9.514047
mAP_valid_abs_values_14	63.0580	9.092967
mAP_valid_abs_values_8	63.0324	7.112824
mAP_valid_abs_values_17	61.0328	6.649189
mAP_valid_abs_values_23	60.5804	6.377480
mAP_valid_abs_values_15	59.9388	3.684010
mAP_valid_abs_values_22	59.7108	1.987400
mAP_valid_abs_values_26	59.1972	2.754980
mAP_valid_abs_values_25	59.0892	2.678659

Model performance on Test data

We then apply the hyperparameters to the test set and average the results.

test_data_1['average_map'] = test_data_1_sub.apply(np.mean, axis=1)

test_data_1.sort_values(by=['average_map'], ascending= False,inplace=True)

	mean	std
mAP_test_abs_values_8	62.393600	5.609268
mAP_test_abs_values_23	62.072400	0.855415
mAP_test_abs_values_13	61.808000	9.617606
mAP_test_abs_values_25	61.674400	2.769447
mAP_test_abs_values_22	61.600400	2.616833
mAP_test_abs_values_21	61.485600	0.936612
mAP_test_abs_values_10	61.449200	8.447344
mAP_test_abs_values_14	61.414000	5.849064
mAP_test_abs_values_26	61.213200	3.157670
mAP_test_abs_values_2	61.106400	0.905386
mAP_test_abs_values_19	60.828400	6.366713
mAP_test_abs_values_15	60.603600	2.701689
mAP_test_abs_values_17	60.565200	0.714540
mAP_test_abs_values_12	60.537600	2.164862
average_map	60.503984	0.819719
mAP_test_abs_values_6	60.434800	6.985568
mAP_test_abs_values_5	60.289600	1.145915
mAP_test_abs_values_4	60.224800	2.745266
mAP_test_abs_values_11	60.064000	6.897371
mAP_test_abs_values_7	59.983600	6.193833
mAP_test_abs_values_3	59.931200	1.734655
mAP_test_abs_values_18	59.284400	3.048217
mAP_test_abs_values_20	59.134000	3.058660
mAP_test_abs_values	58.910000	1.429146
mAP_test_abs_values_9	58.279600	3.964504
mAP_test_abs_values_16	57.311600	4.299832

Model performance on validation data (Mann-Whitney U Test)

Statistical Significance:

Using the global mean maximum row as the reference, we perform the right-tailed t-test to identify significant hyperparameters.

pvalues_mw = []
pvalues_real_mw = []

for _,index in enumerate(dt_valid_sub.index):

  _, p_value_mw = stats.mannwhitneyu(max_row[:-1], dt_valid_sub.loc[index][:-1], alternative='greater')#, method = 'asymptotic')
  pvalues_real_mw.append(p_value_mw)
  pvalues_mw.append((p_value_mw >= 0.05)*1)
  

dt_valid_sub_mw['pvalues_mw'] = pvalues_real_mw
dt_valid_sub_mw['sig_mw'] = pvalues_mw

dt_valid_sub_mw1 = dt_valid_sub_mw.query('sig_mw == 1')


max_indexx = dt_valid_sub_mw1.iloc[:, :-3].mean().idxmax()

#print(sum(pvalues),max_indexx)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_abs_values	mAP_valid_abs_values_2	mAP_valid_abs_values_3	mAP_valid_abs_values_4	mAP_valid_abs_values_5	mAP_valid_abs_values_6	mAP_valid_abs_values_7	mAP_valid_abs_values_8	mAP_valid_abs_values_9	mAP_valid_abs_values_10	mAP_valid_abs_values_11	mAP_valid_abs_values_12	mAP_valid_abs_values_13	mAP_valid_abs_values_14	mAP_valid_abs_values_15	mAP_valid_abs_values_16	mAP_valid_abs_values_17	mAP_valid_abs_values_18	mAP_valid_abs_values_19	mAP_valid_abs_values_20	mAP_valid_abs_values_21	mAP_valid_abs_values_22	mAP_valid_abs_values_23	mAP_valid_abs_values_25	mAP_valid_abs_values_26
1152	372	8	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	68.37	65.62	62.78	62.22	66.23	59.08	55.79	65.66	67.73	66.64	61.94	59.19	65.92	69.33	63.05	58.05	64.53	65.59	48.51	61.12	65.82	66.16	67.21	65.33	65.58
1164	372	8	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	68.20	62.28	67.24	63.71	64.84	62.34	57.31	65.37	66.57	65.00	61.40	60.71	65.61	62.86	65.27	64.89	65.44	80.75	61.27	58.40	62.14	64.95	66.70	68.03	68.45
1167	372	8	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	64.74	63.62	56.51	62.24	61.92	62.08	65.69	61.39	63.12	64.24	60.56	61.79	87.10	60.02	65.89	63.95	61.63	62.28	59.09	59.26	64.43	62.57	64.12	63.12	62.98
1344	372	8	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	71.63	71.01	68.87	72.86	65.75	66.28	52.33	70.72	69.71	69.62	63.73	62.72	67.67	66.51	66.65	75.97	69.87	67.94	43.72	65.17	70.15	69.81	66.60	66.17	70.97
1356	372	8	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	66.02	67.08	63.26	64.44	63.59	62.18	86.68	61.56	66.04	64.95	61.64	60.61	64.28	63.33	65.01	63.63	64.21	64.56	61.17	60.41	69.03	67.75	63.79	66.70	63.47
1359	372	8	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	65.93	63.97	62.87	65.07	66.21	56.35	55.68	65.35	87.18	64.97	60.30	60.50	56.87	64.98	63.06	63.72	63.21	64.10	61.93	59.75	66.79	68.89	62.23	64.56	65.07
576	372	8	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	67.58	66.82	66.14	66.51	74.73	61.83	57.37	67.78	69.95	68.43	62.89	66.38	70.33	65.18	65.88	63.67	68.64	65.64	61.94	62.41	67.30	66.56	69.13	66.34	68.04
579	372	8	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	58.62	62.09	65.54	66.28	61.68	60.81	65.21	61.38	59.34	64.62	59.60	59.97	62.47	59.40	64.30	64.62	64.82	61.93	60.60	59.91	66.04	64.38	64.45	67.52	63.01
588	372	8	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	85.53	62.74	64.39	64.03	86.68	60.35	65.91	62.88	63.17	63.80	60.97	61.54	62.83	61.20	65.91	60.72	60.69	63.47	60.69	58.51	63.84	64.44	62.72	64.53	62.64
771	372	8	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	65.44	61.40	61.10	65.25	64.80	59.64	63.15	63.02	64.78	61.44	60.84	59.71	87.78	62.64	62.34	63.80	62.27	61.76	58.18	59.43	64.48	63.84	62.32	66.20	66.27
780	372	8	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	65.55	65.59	64.78	61.61	64.71	59.20	54.62	64.38	64.49	66.82	63.20	58.30	61.61	61.87	61.80	62.71	61.54	63.85	59.75	59.95	62.30	63.18	63.15	62.75	62.69
783	372	8	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	66.47	64.73	63.32	64.49	64.87	62.01	66.87	64.63	64.15	62.51	59.20	59.03	64.50	62.32	63.84	67.19	65.54	63.40	59.49	57.50	67.09	63.80	68.22	65.74	67.35
960	372	8	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	67.03	64.19	56.95	63.81	60.94	64.67	65.05	62.46	63.43	54.37	56.47	63.51	64.68	63.70	61.45	63.25	61.09	64.53	58.98	63.46	67.46	65.03	63.54	67.04	65.84
972	372	8	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	65.10	67.66	68.14	66.10	64.86	62.40	65.70	65.89	65.97	64.24	60.81	60.81	54.60	64.30	65.57	64.52	65.70	67.77	60.74	61.45	64.57	64.24	60.94	64.34	62.28
15	372	8	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	65.64	65.06	64.95	65.96	64.71	62.16	78.06	57.81	63.14	65.11	63.03	62.97	65.27	64.14	64.50	61.53	63.07	69.06	59.89	59.13	64.92	67.46	65.81	66.75	69.14
207	372	8	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	67.68	68.93	68.88	67.94	67.85	66.44	69.21	64.78	66.59	64.92	67.51	63.61	67.26	64.58	66.57	71.89	69.79	66.18	65.79	63.18	65.28	71.54	67.96	68.13	68.20
387	372	8	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	68.45	66.44	57.87	62.97	57.16	43.72	79.22	78.65	63.73	65.90	62.13	60.09	89.08	60.59	63.28	54.04	63.36	64.43	57.30	61.77	63.84	61.54	64.79	67.58	60.06
396	372	8	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	67.38	63.45	66.94	67.16	65.95	59.19	56.12	64.03	66.74	64.89	57.94	65.81	59.34	86.95	61.57	65.74	64.13	66.61	61.11	66.23	65.37	65.37	62.85	67.57	65.24
1216	372	8	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	63.27	61.16	63.64	63.89	63.04	59.41	87.96	62.02	60.42	60.50	88.80	57.82	90.78	60.42	61.33	62.40	62.97	62.51	84.02	88.94	65.12	61.35	62.71	66.26	62.31
1219	372	8	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	88.66	88.44	62.31	61.96	60.94	91.28	87.20	63.64	89.36	62.82	86.98	90.70	56.04	60.84	61.22	84.67	59.35	61.39	59.15	59.83	63.82	61.67	61.13	63.62	59.53
1231	372	8	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	88.83	62.63	62.88	61.70	61.02	91.52	62.38	60.62	62.78	62.57	91.21	60.65	92.54	83.03	62.55	91.64	60.60	61.60	60.32	57.10	60.35	62.67	62.53	61.87	60.43
1420	372	8	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	58.00	63.55	65.35	64.01	63.21	90.13	66.64	65.16	62.63	61.24	59.39	61.20	89.55	90.74	62.02	60.47	62.25	62.40	59.10	59.11	61.70	61.66	61.13	68.24	60.07
1423	372	8	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	63.70	62.37	62.03	88.84	60.23	61.44	64.98	60.90	62.50	63.05	89.51	60.58	91.12	61.37	61.74	61.97	61.88	60.87	90.22	58.54	62.27	61.39	63.06	60.80	58.31
640	372	8	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	86.78	62.72	64.46	62.61	88.36	59.86	60.86	60.01	63.54	90.09	91.73	57.20	91.82	60.32	60.06	60.89	60.79	63.65	57.56	59.02	63.41	62.58	59.24	63.81	63.14
643	372	8	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	63.48	65.16	64.33	64.38	64.33	89.10	88.51	61.65	43.72	63.83	58.95	60.21	63.52	62.82	63.10	88.23	60.38	84.69	60.37	59.06	62.90	64.29	60.74	60.24	62.88
652	372	8	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	86.67	64.49	61.42	62.08	60.72	90.32	86.21	59.43	88.09	61.24	90.18	59.40	90.28	59.57	62.97	60.40	61.65	58.75	59.69	58.14	59.88	62.49	58.55	60.71	62.26
832	372	8	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	63.11	64.46	88.09	62.35	62.15	62.14	82.81	63.12	49.89	63.73	87.53	61.40	63.48	61.57	62.47	63.82	61.98	61.60	60.14	91.03	64.82	61.79	61.32	59.75	61.32
847	372	8	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	88.46	86.61	61.60	60.41	63.10	59.16	85.88	60.31	55.85	62.92	90.63	58.39	90.68	61.86	60.50	62.60	60.27	63.86	91.04	58.40	61.64	59.96	60.87	61.77	60.26
1024	372	8	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	64.52	61.93	74.37	61.89	63.27	59.53	88.48	60.54	60.39	62.14	59.26	56.64	78.46	89.22	64.01	85.55	59.32	89.74	60.38	56.91	66.98	64.43	59.66	58.94	61.59
1027	372	8	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	64.88	61.51	86.76	60.04	65.18	90.99	87.70	60.12	61.58	43.72	90.08	91.36	61.08	85.45	58.94	56.47	61.12	59.12	90.94	56.10	61.38	60.35	61.98	55.09	61.57
76	372	8	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	64.71	60.70	62.10	87.85	61.14	91.88	88.30	59.57	43.72	87.16	59.27	92.18	84.02	59.93	61.20	88.62	60.33	59.00	90.56	55.89	59.57	60.94	56.45	60.44	61.52
79	372	8	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	88.57	62.53	86.74	61.84	61.35	92.60	88.93	61.47	89.55	62.20	92.17	59.70	84.94	62.20	43.72	60.71	59.85	61.03	59.30	58.30	64.25	62.34	59.50	62.20	58.68
256	372	8	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	86.22	64.25	87.63	63.04	63.92	60.11	88.64	62.74	64.17	87.34	60.43	57.99	87.00	61.27	64.76	92.24	59.90	56.52	59.80	92.46	59.68	60.61	61.01	57.17	59.93
259	372	8	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	64.49	63.25	61.96	61.22	62.59	89.97	88.17	63.16	89.63	62.17	91.99	58.58	61.24	55.73	62.05	46.94	62.82	59.63	59.87	89.60	63.15	63.13	60.36	51.48	59.83
463	372	8	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	89.13	62.47	61.20	62.12	62.08	58.95	87.86	62.03	89.96	62.83	61.02	55.65	43.72	60.36	61.93	92.22	63.77	60.02	58.25	56.71	61.97	57.70	88.11	61.05	60.48
1472	372	8	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	88.72	61.06	86.23	61.50	62.09	60.60	86.75	61.45	90.22	62.58	60.17	56.69	92.51	60.05	59.89	59.00	59.55	91.71	58.66	90.48	64.72	55.75	59.63	61.24	59.41
1487	372	8	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	88.03	64.43	89.80	62.32	89.22	59.10	88.69	59.29	90.67	62.82	92.44	60.30	60.73	60.90	60.02	45.81	61.15	90.98	59.98	56.88	89.43	58.24	57.67	58.24	59.04
704	372	8	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	65.25	62.53	62.98	88.91	63.72	59.18	66.74	61.76	63.52	64.78	91.15	60.24	91.00	61.57	61.56	91.71	61.03	90.47	57.54	91.96	65.10	58.98	60.12	58.59	65.50
908	372	8	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	90.41	89.64	64.51	60.15	88.57	91.93	84.97	59.55	61.58	89.38	87.44	89.31	89.54	58.04	60.00	91.49	57.67	61.25	84.20	55.10	59.92	58.83	59.86	58.21	57.07
911	372	8	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	87.15	61.02	89.21	89.45	89.51	59.47	90.45	61.91	89.72	61.17	91.42	92.42	91.92	91.66	59.65	92.11	61.52	92.24	57.28	59.07	62.80	61.12	58.26	49.09	60.25
1100	372	8	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	87.45	64.21	62.42	62.60	63.56	91.34	89.45	62.28	60.89	60.11	59.25	59.21	92.45	60.52	61.51	90.03	60.77	60.42	92.24	86.88	59.79	60.05	57.43	59.66	49.26
1103	372	8	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	63.03	89.71	63.81	61.33	62.54	91.95	87.41	60.60	61.73	89.97	55.43	60.38	88.64	60.93	60.68	90.69	59.64	59.23	93.43	58.29	56.40	58.01	58.74	59.47	59.36
140	372	8	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	89.47	61.30	61.12	62.20	87.85	90.42	89.08	61.95	60.60	82.15	89.54	57.66	91.40	60.55	61.03	88.85	92.48	60.05	58.38	58.68	57.06	62.07	90.09	59.18	57.38
320	372	8	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	88.33	64.65	62.29	61.26	62.48	57.16	86.62	61.54	90.72	61.19	59.23	43.72	91.18	59.52	61.15	90.49	59.75	91.46	60.88	58.32	59.05	63.79	60.61	60.63	57.94

Size of the All data:  (100, 28)
Size of the Sig data:  (44, 33)

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params_mw1 = dt_valid.loc[dt_valid_sub_mw1.index][['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
optimal_params_mw = optimal_params_mw1.reset_index()
optimal_params_mw.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_mw1 = pd.merge(optimal_params_mw, dt_test, on=['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg'])
test_data_1_sub_mw = test_data_mw1[list(test_data_mw1.columns[test_data_mw1.columns.str.startswith('mAP_test')])]

Equivalent test data

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_abs_values	mAP_test_abs_values_2	mAP_test_abs_values_3	mAP_test_abs_values_4	mAP_test_abs_values_5	mAP_test_abs_values_6	mAP_test_abs_values_7	mAP_test_abs_values_8	mAP_test_abs_values_9	mAP_test_abs_values_10	mAP_test_abs_values_11	mAP_test_abs_values_12	mAP_test_abs_values_13	mAP_test_abs_values_14	mAP_test_abs_values_15	mAP_test_abs_values_16	mAP_test_abs_values_17	mAP_test_abs_values_18	mAP_test_abs_values_19	mAP_test_abs_values_20	mAP_test_abs_values_21	mAP_test_abs_values_22	mAP_test_abs_values_23	mAP_test_abs_values_25	mAP_test_abs_values_26
39	372	8	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	60.37	59.61	62.79	61.30	60.04	58.65	48.61	60.23	60.99	61.33	59.69	59.64	61.55	58.70	61.14	64.98	60.01	59.73	45.95	58.82	61.66	60.99	61.35	61.42	61.67
31	372	8	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	60.29	59.06	60.26	59.11	58.96	59.30	49.40	59.95	57.95	58.74	60.44	59.07	61.23	61.30	59.04	59.70	60.46	59.21	60.62	57.95	62.32	59.21	62.06	60.46	61.03
40	372	8	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	59.77	60.61	50.31	61.56	60.59	59.52	61.64	59.37	61.15	60.78	60.79	59.54	58.83	59.18	61.36	59.91	60.02	59.35	59.66	59.65	60.04	61.29	61.47	61.44	61.43
20	372	8	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	58.91	58.94	61.11	58.72	61.35	56.96	50.01	59.52	59.68	60.07	58.32	57.76	60.52	60.60	61.13	61.70	59.03	61.34	48.12	57.87	59.88	60.80	61.63	61.83	62.05
29	372	8	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	59.76	60.13	60.05	60.35	60.05	59.96	57.98	59.20	60.40	60.34	62.12	59.63	59.23	59.66	60.04	60.79	59.02	58.91	60.55	59.14	60.72	60.68	61.87	60.69	61.55
34	372	8	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	59.03	61.01	60.33	60.32	61.74	49.79	49.14	59.19	56.83	59.25	62.16	60.12	49.87	59.80	59.64	60.08	59.62	59.50	59.18	59.86	59.54	59.35	61.98	61.48	61.16
22	372	8	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	60.45	61.30	59.49	60.16	61.74	60.65	47.99	59.83	59.84	59.66	58.99	59.42	59.25	65.80	60.63	57.96	59.14	58.87	59.85	59.29	61.48	60.29	60.60	61.10	60.83
42	372	8	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	48.54	60.64	59.58	61.28	59.33	60.09	59.38	60.90	49.46	61.36	60.48	59.51	61.29	59.35	59.93	58.57	60.40	59.43	60.17	59.24	60.95	61.78	60.88	60.48	61.94
33	372	8	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	56.41	61.73	59.97	59.28	58.14	59.05	60.52	59.74	58.57	59.96	59.29	61.46	59.41	59.37	60.99	60.52	59.27	59.80	60.03	58.47	60.92	62.67	62.42	62.53	62.51
38	372	8	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	60.37	60.53	59.30	59.82	59.35	59.11	58.86	59.02	59.90	58.47	60.42	58.41	57.62	59.61	60.41	59.59	59.15	58.18	58.40	58.78	61.00	62.25	62.75	61.69	61.80
43	372	8	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	60.00	61.22	59.12	59.67	58.37	58.81	50.37	61.32	59.24	60.53	60.17	59.41	60.04	59.35	60.91	59.38	59.46	59.86	59.17	59.99	62.31	60.74	61.62	61.28	62.22
36	372	8	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	60.56	59.71	59.33	60.97	60.61	59.52	60.07	60.49	58.70	60.13	59.75	59.84	59.18	60.14	59.75	60.20	58.81	59.11	58.85	58.62	60.71	61.40	60.08	61.70	60.09
41	372	8	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	60.48	58.84	49.00	61.14	60.04	60.17	61.04	58.86	60.07	50.20	50.66	60.31	58.74	60.01	59.48	59.73	62.43	58.82	59.54	59.23	61.64	62.09	62.32	67.02	61.17
35	372	8	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	59.24	59.36	60.97	61.00	60.42	59.84	61.00	64.43	60.39	59.34	59.26	59.12	52.63	59.28	61.40	58.31	59.07	58.26	60.52	60.05	61.14	61.89	62.95	60.19	61.50
32	372	8	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	60.25	58.63	60.50	60.99	59.26	60.35	49.61	48.88	60.44	60.66	60.22	58.65	61.05	58.78	59.94	59.15	60.67	59.97	59.96	59.63	61.82	61.43	61.69	61.15	60.55
17	372	8	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	60.28	60.82	59.19	60.77	60.90	60.16	59.94	61.04	60.33	60.56	60.48	60.03	60.91	61.07	60.33	57.46	58.94	60.43	60.18	59.77	61.13	59.88	60.71	61.06	60.09
37	372	8	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	60.28	60.99	49.50	59.37	50.77	48.12	49.53	34.94	60.22	60.52	59.73	59.64	57.88	59.10	60.57	62.90	59.64	59.94	58.51	59.68	61.56	60.80	60.91	60.38	62.57
30	372	8	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	59.09	60.24	61.49	60.70	59.90	48.56	50.43	59.64	58.99	59.19	58.53	59.02	46.16	59.23	59.32	58.50	60.31	60.21	58.66	58.28	61.41	62.22	61.58	60.97	61.89
16	372	8	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	60.29	58.88	58.27	59.47	60.96	58.90	58.23	59.86	58.78	60.37	51.23	59.47	58.42	58.87	59.18	61.58	60.81	62.28	87.45	56.55	61.40	61.39	61.63	61.38	61.64
3	372	8	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	58.28	59.36	61.75	60.33	60.39	59.30	58.88	61.50	58.12	61.65	50.92	59.48	50.25	60.22	61.29	52.21	60.78	59.76	59.47	61.65	61.61	62.72	61.79	61.42	61.71
11	372	8	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	58.67	60.27	60.46	61.03	59.32	60.02	61.50	61.64	59.92	61.77	59.33	61.59	59.18	86.26	59.51	59.70	60.43	61.45	61.53	59.63	63.13	61.59	62.18	61.63	62.10
28	372	8	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	50.11	61.27	58.98	61.63	60.02	58.25	59.78	61.15	60.09	59.62	62.13	61.93	58.32	59.24	60.30	59.93	60.67	60.68	61.03	60.19	61.77	62.31	62.36	60.26	62.08
23	372	8	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	61.92	61.03	61.37	57.93	60.93	61.45	60.20	60.77	61.42	61.28	55.55	61.59	58.63	61.33	60.50	60.24	60.29	60.38	57.35	61.53	60.75	62.25	62.11	61.45	61.00
18	372	8	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	57.99	60.21	59.58	61.00	57.73	60.33	58.21	59.95	61.74	58.42	59.64	58.78	58.51	60.82	60.46	59.90	60.54	60.27	59.71	60.40	61.68	62.03	62.78	61.74	61.32
27	372	8	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	61.36	60.61	61.13	61.09	61.63	59.73	58.49	61.39	48.12	61.11	60.42	60.58	60.42	60.41	59.95	57.77	60.18	82.74	60.06	60.70	60.98	61.74	61.66	62.07	60.83
15	372	8	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	56.37	61.66	60.05	61.79	59.96	57.62	56.24	61.40	58.62	61.16	59.15	61.47	58.97	60.51	61.25	60.77	60.48	60.63	61.58	59.21	62.44	61.45	61.79	61.67	61.68
25	372	8	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	60.48	61.67	58.67	59.36	61.23	60.98	58.96	60.32	53.08	61.16	47.75	61.53	60.64	60.57	60.71	59.77	60.35	60.99	62.28	58.05	61.06	62.44	62.17	61.79	62.48
14	372	8	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	58.93	57.38	60.72	58.88	60.38	60.76	56.88	61.05	48.85	61.35	59.16	60.53	58.76	60.47	60.43	59.71	59.80	61.07	58.03	60.54	62.28	60.86	60.80	61.32	60.97
21	372	8	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	61.27	61.25	52.36	59.48	61.44	62.71	57.80	59.08	60.15	60.77	60.37	50.57	47.44	58.54	57.87	50.54	58.64	56.67	60.82	58.25	61.46	60.99	62.49	62.74	61.97
13	372	8	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	60.53	59.99	58.64	60.24	60.74	59.67	58.80	59.26	58.73	48.12	50.01	59.15	59.59	47.94	60.59	49.59	60.45	60.70	58.70	59.60	60.63	62.15	60.19	62.50	61.08
7	372	8	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	60.95	59.58	61.08	57.94	60.60	59.52	57.26	59.78	48.12	56.81	62.29	58.54	85.89	60.97	60.67	57.43	60.06	59.56	49.26	59.33	61.27	62.52	63.39	61.86	61.22
12	372	8	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	59.05	61.76	55.83	60.60	60.28	58.98	59.89	60.55	57.15	60.13	59.88	62.70	87.16	60.16	48.12	60.31	59.74	60.08	60.77	61.29	60.92	62.25	62.69	62.20	61.64
6	372	8	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	57.17	61.06	58.92	61.99	60.47	60.84	58.17	62.50	60.99	58.02	62.78	59.25	57.89	60.58	60.95	59.02	60.57	50.47	60.48	59.35	61.95	62.78	62.90	63.48	62.10
24	372	8	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	61.16	60.37	61.35	60.33	61.22	58.91	57.47	61.67	57.69	59.51	57.43	61.43	61.07	49.19	61.04	75.37	59.66	61.18	60.09	59.57	61.78	62.14	61.94	45.10	61.96
26	372	8	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	57.99	60.64	60.62	62.36	60.89	58.77	57.33	60.08	58.88	60.71	61.62	58.59	48.12	59.63	60.71	59.56	61.02	60.08	61.37	59.38	62.26	62.76	60.03	62.20	61.50
10	372	8	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	58.88	62.32	56.32	60.97	61.41	60.92	88.18	61.11	57.61	60.48	62.32	59.93	58.31	62.18	61.58	60.71	60.72	58.94	60.46	50.61	60.25	49.51	62.36	62.52	61.00
4	372	8	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	59.74	61.52	59.26	62.32	59.41	62.08	58.77	61.93	59.10	61.42	59.79	62.15	62.16	61.21	61.42	65.45	60.75	59.48	61.53	59.45	60.03	60.47	61.95	61.82	62.28
5	372	8	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	61.16	61.87	61.27	59.81	61.81	60.11	59.64	61.76	60.67	61.39	60.33	62.39	59.28	60.90	61.75	59.43	61.49	59.45	62.97	59.04	60.33	62.55	62.20	62.42	61.84
1	372	8	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	57.80	59.56	61.33	60.44	58.48	59.95	51.41	62.00	61.36	59.18	51.55	51.81	60.01	61.70	62.25	59.72	61.00	61.46	87.31	59.06	61.43	61.60	61.15	62.61	62.49
0	372	8	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	59.59	62.00	59.07	59.52	59.22	61.90	59.49	60.91	59.71	62.32	60.22	59.95	59.59	58.21	62.28	59.05	60.03	59.22	62.90	60.65	61.57	62.42	62.09	48.62	62.01
8	372	8	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	59.69	61.55	61.21	61.31	60.72	59.48	59.22	61.70	61.51	61.44	61.87	61.58	59.52	61.48	61.70	57.64	60.82	62.09	59.37	52.38	62.90	62.33	62.28	62.56	46.25
9	372	8	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	61.88	59.48	61.29	60.75	61.99	58.62	59.27	61.15	61.02	59.12	59.17	61.99	60.19	60.59	60.40	59.37	61.36	60.98	59.95	61.87	62.48	62.69	62.56	62.08	62.25
2	372	8	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	59.00	61.40	61.28	60.90	59.80	59.80	59.23	60.69	60.77	82.94	49.59	60.17	59.73	61.43	61.99	48.48	58.81	61.99	61.25	61.42	61.60	61.82	59.47	61.39	62.45
19	372	8	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	58.01	61.29	61.62	61.59	60.77	60.06	56.64	61.70	58.40	61.11	61.19	48.12	58.59	60.72	60.79	58.74	60.57	58.51	61.65	61.36	61.92	61.68	62.40	62.50	62.12

Size of the test data:  (44, 33)

Difference

diff_mw = dt_mw[['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
for i, cols in enumerate(zip(dt_mw.columns,test_data_mw1.columns)):   
   diff_mw[f'col_diff_{i}'] = (np.array(dt_mw[cols[0]]) - np.array(test_data_mw1[cols[1]]))*-1

# Function to count negatives in a row
def count_negatives(row):
    return sum(1 for value in row if isinstance(value, (int, float)) and value < 0)

diff_mw['negative_count'] = diff_mw.apply(count_negatives, axis=1)

diff_mw.sort_values(by =["negative_count"],inplace = True, ascending=False)


html_table = diff_mw.to_html(index=True)

# Wrap in a scrollable div
scrollable_table = f"""
<div style="height: 400px; width: 100%; overflow-x: auto; overflow-y: auto;">
    {html_table}
</div>
"""
# Display the scrollable table
display(HTML(scrollable_table))

key_values = ['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']
print('Are the keys of the valid and test dfs same?: ',dt_mw[key_values].equals(test_data_mw1[key_values]))

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	col_diff_0	col_diff_1	col_diff_2	col_diff_3	col_diff_4	col_diff_5	col_diff_6	col_diff_7	col_diff_8	col_diff_9	col_diff_10	col_diff_11	col_diff_12	col_diff_13	col_diff_14	col_diff_15	col_diff_16	col_diff_17	col_diff_18	col_diff_19	col_diff_20	col_diff_21	col_diff_22	col_diff_23	col_diff_24	col_diff_25	col_diff_26	col_diff_27	col_diff_28	col_diff_29	col_diff_30	col_diff_31	col_diff_32	negative_count
588	372	8	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.12	-1.01	-4.42	-4.75	-28.54	-1.30	-5.39	-3.14	-4.60	-3.84	-1.68	-0.08	-3.42	-1.83	-4.92	-0.20	-1.42	-3.67	-0.66	-0.04	-2.92	-1.77	-0.30	-2.00	-0.13	25
207	372	8	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-7.40	-8.11	-9.69	-7.17	-6.95	-6.28	-9.27	-3.74	-6.26	-4.36	-7.03	-3.58	-6.35	-3.51	-6.24	-14.43	-10.85	-5.75	-5.61	-3.41	-4.15	-11.66	-7.25	-7.07	-8.11	25
1164	372	8	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-7.91	-3.22	-6.98	-4.60	-5.88	-3.04	-7.91	-5.42	-8.62	-6.26	-0.96	-1.64	-4.38	-1.56	-6.23	-5.19	-4.98	-21.54	-0.65	-0.45	0.18	-5.74	-4.64	-7.57	-7.42	24
1344	372	8	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-12.72	-12.07	-7.76	-14.14	-4.40	-9.32	-2.32	-11.20	-10.03	-9.55	-5.41	-4.96	-7.15	-5.91	-5.52	-14.27	-10.84	-6.60	4.40	-7.30	-10.27	-9.01	-4.97	-4.34	-8.92	24
1356	372	8	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-6.26	-6.95	-3.21	-4.09	-3.54	-2.22	-28.70	-2.36	-5.64	-4.61	0.48	-0.98	-5.05	-3.67	-4.97	-2.84	-5.19	-5.65	-0.62	-1.27	-8.31	-7.07	-1.92	-6.01	-1.92	24
576	372	8	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-7.13	-5.52	-6.65	-6.35	-12.99	-1.18	-9.38	-7.95	-10.11	-8.77	-3.90	-6.96	-11.08	0.62	-5.25	-5.71	-9.50	-6.77	-2.09	-3.12	-5.82	-6.27	-8.53	-5.24	-7.21	24
579	372	8	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-10.08	-1.45	-5.96	-5.00	-2.35	-0.72	-5.83	-0.48	-9.88	-3.26	0.88	-0.46	-1.18	-0.05	-4.37	-6.05	-4.42	-2.50	-0.43	-0.67	-5.09	-2.60	-3.57	-7.04	-1.07	24
396	372	8	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-8.29	-3.21	-5.45	-6.46	-6.05	-10.63	-5.69	-4.39	-7.75	-5.70	0.59	-6.79	-13.18	-27.72	-2.25	-7.24	-3.82	-6.40	-2.45	-7.95	-3.96	-3.15	-1.27	-6.60	-3.35	24
972	372	8	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-5.86	-8.30	-7.17	-5.10	-4.44	-2.56	-4.70	-1.46	-5.58	-4.90	-1.55	-1.69	-1.97	-5.02	-4.17	-6.21	-6.63	-9.51	-0.22	-1.40	-3.43	-2.35	2.01	-4.15	-0.78	24
960	372	8	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-6.55	-5.35	-7.95	-2.67	-0.90	-4.50	-4.01	-3.60	-3.36	-4.17	-5.81	-3.20	-5.94	-3.69	-1.97	-3.52	1.34	-5.71	0.56	-4.23	-5.82	-2.94	-1.22	-0.02	-4.67	23
15	372	8	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-5.39	-6.43	-4.45	-4.97	-5.45	-1.81	-28.45	-8.93	-2.70	-4.45	-2.81	-4.32	-4.22	-5.36	-4.56	-2.38	-2.40	-9.09	0.07	0.50	-3.10	-6.03	-4.12	-5.60	-8.59	23
1359	372	8	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-6.90	-2.96	-2.54	-4.75	-4.47	-6.56	-6.54	-6.16	-30.35	-5.72	1.86	-0.38	-7.00	-5.18	-3.42	-3.64	-3.59	-4.60	-2.75	0.11	-7.25	-9.54	-0.25	-3.08	-3.91	23
771	372	8	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-5.07	-0.87	-1.80	-5.43	-5.45	-0.53	-4.29	-4.00	-4.88	-2.97	-0.42	-1.30	-30.16	-3.03	-1.93	-4.21	-3.12	-3.58	0.22	-0.65	-3.48	-1.59	0.43	-4.51	-4.47	23
1216	372	8	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-2.98	-2.28	-5.37	-4.42	-2.08	-0.51	-29.73	-2.16	-1.64	-0.13	-37.57	1.65	-32.36	-1.55	-2.15	-0.82	-2.16	-0.23	3.43	-32.39	-3.72	0.04	-1.08	-4.88	-0.67	22
1152	372	8	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-8.00	-6.01	0.01	-0.92	-6.19	-0.43	-7.18	-5.43	-6.74	-5.31	-2.25	0.45	-4.37	-10.63	-1.91	6.93	-4.52	-5.86	-2.56	-2.30	-4.16	-5.17	-5.86	-3.91	-3.91	22
783	372	8	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-5.91	-5.02	-3.99	-3.52	-4.26	-2.49	-6.80	-4.14	-5.45	-2.38	0.55	0.81	-5.32	-2.18	-4.09	-6.99	-6.73	-4.29	-0.64	1.12	-6.38	-2.40	-8.14	-4.04	-7.26	22
780	372	8	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-5.55	-4.37	-5.66	-1.94	-6.34	-0.39	-4.25	-3.06	-5.25	-6.29	-3.03	1.11	-1.57	-2.52	-0.89	-3.33	-2.08	-3.99	-0.58	0.04	0.01	-2.44	-1.53	-1.47	-0.47	22
1167	372	8	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-4.97	-3.01	-6.20	-0.68	-1.33	-2.56	-4.05	-2.02	-1.97	-3.46	0.23	-2.25	-28.27	-0.84	-4.53	-4.04	-1.61	-2.93	0.57	0.39	-4.39	-1.28	-2.65	-1.68	-1.55	22
387	372	8	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-8.17	-5.45	-8.37	-3.60	-6.39	4.40	-29.69	-43.71	-3.51	-5.38	-2.40	-0.45	-31.20	-1.49	-2.71	8.86	-3.72	-4.49	1.21	-2.09	-2.28	-0.74	-3.88	-7.20	2.51	21
259	372	8	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-3.33	-2.88	-0.61	-0.89	-1.37	-31.06	-30.70	-1.49	-31.94	-2.66	-34.56	2.85	-0.17	-6.54	-1.01	28.43	-3.16	1.55	0.22	-30.03	-1.37	-0.99	1.58	-6.38	2.13	19
643	372	8	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-2.12	-4.55	-3.20	-3.29	-2.70	-29.37	-30.02	-0.26	4.40	-2.72	1.47	0.37	-3.10	-2.41	-3.15	-30.46	-0.20	-1.95	-0.31	1.64	-1.92	-2.55	0.92	1.83	-2.05	19
832	372	8	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-2.63	-2.79	-29.42	-2.99	-0.92	-1.16	-23.85	-2.80	3.19	-2.57	-39.78	0.13	-2.84	-1.00	-1.76	-4.05	-1.63	-0.61	2.14	-32.98	-3.76	0.65	0.85	2.04	1.16	18
1027	372	8	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-4.35	-1.52	-28.12	0.20	-4.44	-31.32	-28.90	-0.86	-2.85	4.40	-40.07	-32.21	-1.49	-37.51	1.65	-6.88	-0.67	1.58	-32.24	3.50	-0.75	1.80	-1.79	7.41	-0.49	18
1024	372	8	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-3.25	-0.68	-22.01	-2.41	-1.83	3.18	-30.68	-1.46	-0.24	-1.37	1.11	-6.07	-31.02	-30.68	-6.14	-35.01	-0.68	-33.07	0.44	1.34	-5.52	-3.44	2.83	3.80	0.38	18
847	372	8	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.53	-29.23	-0.88	-1.53	-2.72	1.60	-29.00	0.74	-7.00	-1.57	-31.47	2.14	-31.92	-1.39	-0.07	-2.89	-0.47	-2.79	-33.01	2.14	0.64	0.90	-0.07	-0.45	0.71	18
1423	372	8	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-1.78	-1.34	-0.66	-30.91	0.70	0.01	-4.78	-0.13	-1.08	-1.77	-33.96	1.01	-32.49	-0.04	-1.24	-1.73	-1.59	-0.49	-32.87	2.99	-1.52	0.86	-0.95	0.65	2.69	18
1231	372	8	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.16	-2.36	-2.42	-0.67	-1.70	-31.50	-0.88	1.02	-2.86	-0.80	-31.88	0.94	-33.36	3.23	-3.04	-31.94	-0.17	-0.15	1.21	2.53	2.78	-1.08	-0.35	-0.24	1.67	18
1219	372	8	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.38	-29.08	-0.56	-1.63	-0.55	-31.98	-28.32	-2.14	-31.24	-1.17	-36.06	-31.22	-5.79	-0.62	0.07	-32.46	1.43	-1.63	0.32	1.82	-2.21	1.05	0.66	-2.20	2.18	18
640	372	8	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.79	-2.51	-4.88	-1.61	-30.63	0.47	-2.65	-0.06	-1.80	-31.67	-32.09	1.58	-33.31	0.50	0.40	-0.99	-0.25	-3.38	2.15	1.38	-1.73	-0.55	3.54	-2.07	-1.82	18
1420	372	8	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-7.89	-2.28	-6.37	-2.38	-3.19	-31.88	-6.86	-4.01	-2.54	-1.62	2.74	0.73	-31.23	-31.50	-1.72	-0.54	-1.58	-1.72	1.93	1.08	0.07	0.65	1.23	-7.98	2.01	17
79	372	8	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.52	-0.77	-30.91	-1.24	-1.07	-33.62	-29.04	-0.92	-32.40	-2.07	-32.29	3.00	2.22	-2.04	4.40	-0.40	-0.11	-0.95	1.47	2.99	-3.33	-0.09	3.19	-0.00	2.96	17
704	372	8	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-4.09	-0.66	-1.71	-29.10	-1.91	0.93	-7.10	-0.00	-2.85	-3.39	-30.82	2.15	-31.72	-0.67	0.19	-32.28	0.46	-31.02	5.43	-32.92	-4.77	3.57	2.08	3.83	-3.66	16
911	372	8	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.56	0.98	-30.14	-29.93	-30.29	2.43	-30.96	-1.00	-30.01	1.15	-31.20	-32.47	-32.33	-33.45	2.63	-33.06	-1.49	-33.02	5.62	1.58	-1.23	1.30	3.83	-0.47	1.76	16
652	372	8	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.30	-2.83	-1.37	-0.29	-0.76	-32.70	-29.97	1.97	-29.47	-0.08	-31.03	2.07	-31.31	0.94	-1.72	0.37	-1.17	1.88	1.89	1.07	2.56	-1.04	3.24	0.96	-0.58	15
256	372	8	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.05	-3.19	-28.71	-1.05	-3.45	0.73	-30.47	-0.24	-3.18	-29.32	2.35	1.26	-29.11	-0.69	-3.81	-33.22	0.67	-6.05	0.68	-33.11	2.27	2.17	1.89	6.31	2.17	15
76	372	8	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-3.76	-1.12	-1.02	-29.91	-0.54	-32.36	-31.04	0.21	4.40	-30.35	3.02	-33.64	1.87	1.04	-0.53	-31.19	-0.27	0.56	-41.30	3.44	1.70	1.58	6.94	1.42	-0.30	14
463	372	8	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-31.14	-1.83	-0.58	0.24	-1.19	-0.18	-30.53	-1.95	-31.08	-2.12	0.60	2.94	4.40	-0.73	-1.22	-32.66	-2.75	0.06	3.12	2.67	0.29	5.06	-28.08	1.15	1.02	14
1103	372	8	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-1.15	-30.23	-2.52	-0.58	-0.55	-33.33	-28.14	0.55	-0.71	-30.85	3.74	1.61	-28.45	-0.34	-0.28	-31.32	1.72	1.75	-33.48	3.58	6.08	4.68	3.82	2.61	2.89	14
1100	372	8	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.76	-2.66	-1.21	-1.29	-2.84	-31.86	-30.23	-0.58	0.62	1.33	2.62	2.37	-32.93	0.96	0.19	-32.39	0.05	1.67	-32.87	-34.50	3.11	2.28	4.85	2.90	-3.01	13
140	372	8	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.47	0.10	0.16	-1.30	-28.05	-30.62	-29.85	-1.26	0.17	0.79	-39.95	2.51	-31.67	0.88	0.96	-40.37	-33.67	1.94	2.87	2.74	4.54	-0.25	-30.62	2.21	5.07	12
1472	372	8	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.84	1.26	-29.91	-0.53	-0.68	0.32	1.43	-0.34	-32.61	-2.10	2.15	3.24	-34.20	2.13	1.69	1.71	1.17	-32.77	1.80	-39.87	-4.47	-6.24	2.73	1.28	1.59	12
908	372	8	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-32.61	-30.08	-3.18	0.29	-30.09	-31.98	-33.56	2.45	-0.22	-30.20	-35.89	-37.50	-29.53	3.66	2.25	-31.77	3.33	0.21	3.11	3.96	1.51	2.77	1.29	4.40	5.42	12
320	372	8	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.32	-3.36	-0.67	0.33	-1.71	2.90	-29.98	0.16	-32.32	-0.08	1.96	4.40	-32.59	1.20	-0.36	-31.75	0.82	-32.95	0.77	3.04	2.87	-2.11	1.79	1.87	4.18	12
1487	372	8	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.29	-2.91	-30.54	-0.00	-29.81	2.98	-29.92	2.64	-31.57	-1.40	-32.65	1.85	1.43	0.31	1.40	19.64	-0.40	-31.50	1.55	2.57	-29.40	2.23	4.28	3.58	3.24	11

Are the keys of the valid and test dfs same?:  False

Model performance on valid data

Average & Std Deviation of the Significant rows:

	mean	std
mAP_valid_abs_values_13	75.610000	14.544143
mAP_valid_abs_values_7	75.491818	13.203204
mAP_valid_abs_values	73.560909	11.303586
mAP_valid_abs_values_11	71.349091	14.550901
mAP_valid_abs_values_16	70.770682	14.126503
mAP_valid_abs_values_6	69.324318	14.580743
mAP_valid_abs_values_9	68.255909	12.773298
mAP_valid_abs_values_3	67.494773	9.450812
mAP_valid_abs_values_18	67.455455	10.726516
mAP_valid_abs_values_5	66.901136	9.013690
mAP_valid_abs_values_10	66.276591	9.462459
mAP_valid_abs_values_2	66.124091	7.519382
mAP_valid_abs_values_4	65.740909	7.734426
mAP_valid_abs_values_14	65.325909	9.396410
mAP_valid_abs_values_19	64.623864	12.420444
mAP_valid_abs_values_20	64.269091	11.644751
mAP_valid_abs_values_21	64.045682	4.926097
mAP_valid_abs_values_23	63.347500	6.435589
mAP_valid_abs_values_12	63.337045	10.618225
mAP_valid_abs_values_17	63.007273	5.282553
mAP_valid_abs_values_22	62.902273	3.264319
mAP_valid_abs_values_8	62.830909	3.452881
mAP_valid_abs_values_25	62.412500	4.383406
mAP_valid_abs_values_26	62.195455	3.936205
mAP_valid_abs_values_15	62.181818	3.496953

Model performance on Test data

We then apply the hyperparameters to the test set and average the results.

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params = dt_valid.loc[dt_valid_sub_mw1.index].iloc[:, :6].reset_index()
optimal_params.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_1 = pd.merge(optimal_params, dt_test, how='left')
test_data_1_sub = test_data_1[list(test_data_1.columns[test_data_1.columns.str.startswith('mAP_test')])]
test_data_1['average_map'] = test_data_1_sub.apply(np.mean, axis=1)

test_data_1.sort_values(by=['average_map'], ascending= False,inplace=True)

	mean	std
mAP_test_abs_values_21	61.729432	2.681800
mAP_test_abs_values_26	61.439943	1.789790
mAP_test_abs_values_25	61.346591	2.705651
mAP_test_abs_values_23	61.335909	3.009567
mAP_test_abs_values_22	61.173864	2.546678
mAP_test_abs_values_18	60.340170	4.630166
mAP_test_abs_values_4	60.209659	1.771109
mAP_test_abs_values_15	60.174943	2.057500
mAP_test_abs_values_14	60.164091	4.441917
mAP_test_abs_values_8	60.134943	4.750951
mAP_test_abs_values_2	60.115057	2.394050
mAP_test_abs_values_10	59.947045	3.897951
mAP_test_abs_values_12	59.803977	2.197745
average_map	59.742191	0.969075
mAP_test_abs_values_5	59.733693	2.197233
mAP_test_abs_values_17	59.720000	2.608334
mAP_test_abs_values_19	59.700170	5.950423
mAP_test_abs_values_20	59.626648	4.476815
mAP_test_abs_values_3	58.929205	3.265148
mAP_test_abs_values_11	58.850341	3.367100
mAP_test_abs_values_16	58.743125	4.123951
mAP_test_abs_values_6	58.700795	5.090062
mAP_test_abs_values_13	58.582898	6.210478
mAP_test_abs_values_9	58.525511	4.544155
mAP_test_abs_values	58.212216	4.892959
mAP_test_abs_values_7	56.314545	5.880173

Summary using radar plot

def extract_number(text):
    if isinstance(text, str):
        matches = re.findall(r'\d+', text)
        return int(matches[0]) if matches else 1
    return 1

res_valid1['id'] = res_valid1.index.to_series().apply(extract_number)
res_test1['id'] = res_test1.index.to_series().apply(extract_number)



res_comb = pd.concat([res_valid1,res_test1])
index_series = res_comb.index.to_series()

res_comb['type'] = np.where(
    index_series.str.contains('valid', case=False), 'valid',
    np.where(index_series.str.contains('test', case=False), 'test', 'unknown')
)
res_comb = res_comb.query("type !='unknown'")

res_comb =  res_comb.reset_index(drop=True)


res_test1 = res_test1.sort_values(by=['id']).reset_index().query("index !='average_map'")
data_range1 = np.array(list(res_test1['mean']) + list(res_valid1['mean']))
categories = [str(i) for i in range(1,26)]
fig = go.Figure()

# Valid
fig.add_trace(go.Scatterpolar(
       r=list(res_test1['mean']),
      theta=categories,
      #fill='toself',
      name='Test'
))

# Test
fig.add_trace(go.Scatterpolar(
      r=list(res_valid1['mean']),
      theta=categories,
      #fill='toself',
      name='Valid'
))

# Customization of chart
fig.update_layout(
  title={
      'text': "T-Test",
        'xanchor': 'center',
      'yanchor': 'top'
  },
  polar=dict(
    radialaxis=dict(
      visible=True,
      range=[round(data_range1.min()), round(data_range1.max()) + 1 + 1]
    )),
  showlegend=True
)

fig.show()

##############


res_valid2['id'] = res_valid2.index.to_series().apply(extract_number)
res_test2['id'] = res_test2.index.to_series().apply(extract_number)



res_comb = pd.concat([res_valid2,res_test2])
index_series = res_comb.index.to_series()

res_comb['type'] = np.where(
    index_series.str.contains('valid', case=False), 'valid',
    np.where(index_series.str.contains('test', case=False), 'test', 'unknown')
)
res_comb = res_comb.query("type !='unknown'")

res_comb =  res_comb.reset_index(drop=True)


res_test2 = res_test2.sort_values(by=['id']).reset_index().query("index !='average_map'")

data_range2 = np.array(list(res_test2['mean']) + list(res_valid2['mean']))

categories = [str(i) for i in range(1,26)]
fig = go.Figure()

# Valid
fig.add_trace(go.Scatterpolar(
       r=list(res_test2['mean']),
      theta=categories,
      #fill='toself',
      name='Test'
))

# Test
fig.add_trace(go.Scatterpolar(
      r=list(res_valid2['mean']),
      theta=categories,
      #fill='toself',
      name='Valid'
))

# Customization of chart
fig.update_layout(
  title={
      'text': "MU-Test",
         'xanchor': 'center',
      'yanchor': 'top'
  },
  polar=dict(
    radialaxis=dict(
      visible=True,
      range=[round(data_range2.min()), round(data_range2.max()) + 1]
    )),
  showlegend=True
)

fig.show()

Experiment 2 Results

• In these experimental results, the thresholding is based on a fixed value of 0. This decision is informed by the fact that the binary-like hash values are symmetrically distributed between -1 and 1.

Validation Data mAP Results Preview:

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_zero	mAP_valid_zero_2	mAP_valid_zero_3	mAP_valid_zero_4	mAP_valid_zero_5	mAP_valid_zero_6	mAP_valid_zero_7	mAP_valid_zero_8	mAP_valid_zero_9	mAP_valid_zero_10	mAP_valid_zero_11	mAP_valid_zero_12	mAP_valid_zero_13	mAP_valid_zero_14	mAP_valid_zero_15	mAP_valid_zero_16	mAP_valid_zero_17	mAP_valid_zero_18	mAP_valid_zero_19	mAP_valid_zero_20	mAP_valid_zero_21	mAP_valid_zero_22	mAP_valid_zero_23	mAP_valid_zero_25	mAP_valid_zero_26
0	372	8	0.1	16	0.00100	24	0.000000e+00	0.000000e+00	64.86	63.05	63.35	63.00	52.78	59.21	63.32	60.91	60.60	61.69	60.76	64.31	58.08	58.70	62.38	58.70	61.58	63.72	60.14	63.00	67.91	60.75	60.67	62.61	65.72
3	372	8	0.1	16	0.00100	24	0.000000e+00	1.000000e-08	54.76	57.69	62.24	62.66	61.73	57.74	63.58	60.10	64.86	60.03	57.59	56.79	60.53	56.26	60.94	43.72	63.32	60.17	57.83	56.33	64.19	64.86	60.32	57.79	58.81
12	372	8	0.1	16	0.00100	24	1.000000e-08	0.000000e+00	56.32	43.72	69.38	64.57	62.54	62.11	57.69	62.84	66.21	58.12	43.72	61.99	61.92	64.66	63.93	61.36	63.93	60.34	62.02	57.77	60.34	43.72	65.40	63.95	63.25
15	372	8	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	65.42	65.02	64.94	65.93	64.71	61.95	78.06	57.81	63.08	65.07	62.43	62.91	65.24	64.06	64.44	61.44	62.85	68.90	59.89	59.04	64.92	67.46	65.79	66.72	68.90
64	372	8	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	54.06	43.72	87.33	63.25	62.86	55.95	88.30	58.64	63.64	65.12	58.69	59.83	89.95	55.99	60.41	60.30	60.33	59.94	56.73	59.15	66.14	68.13	61.54	61.95	57.87
67	372	8	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	85.92	62.52	86.84	61.52	60.48	43.72	56.28	61.21	87.16	64.16	88.25	58.68	61.32	58.74	59.05	89.89	61.15	87.99	92.21	56.04	60.29	60.23	59.11	59.73	59.51
76	372	8	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	64.68	60.43	62.10	87.54	61.14	91.50	87.96	59.34	43.72	87.16	59.27	55.86	83.03	59.93	61.03	88.58	60.33	59.00	90.56	55.85	59.44	60.29	56.27	60.43	61.35
79	372	8	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	88.42	62.49	86.25	61.84	61.35	92.42	88.26	61.40	89.55	62.08	92.09	59.67	84.68	61.43	43.72	60.59	59.85	61.00	59.26	58.30	63.92	62.27	59.44	62.05	58.68
128	372	8	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	62.01	61.34	60.30	61.36	63.42	56.15	60.31	60.65	87.72	60.90	59.58	57.69	88.98	59.80	59.75	91.95	59.32	60.54	56.38	57.89	58.39	58.41	57.29	59.23	57.44
131	372	8	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	88.80	61.63	86.18	61.20	62.11	59.05	88.19	51.46	56.17	55.28	58.69	57.99	90.41	60.19	58.68	60.03	59.19	88.77	59.30	88.92	57.61	59.94	57.11	58.98	58.37
140	372	8	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	89.47	61.30	61.12	61.95	87.38	90.42	88.83	61.92	60.35	81.76	88.96	57.45	91.40	60.55	60.78	88.80	92.48	60.05	58.25	58.30	56.84	62.01	89.98	59.06	57.26
143	372	8	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	60.64	64.23	61.81	88.72	59.45	59.28	88.60	59.60	60.91	62.62	91.43	59.09	90.84	60.33	60.39	84.68	57.49	89.93	43.72	58.65	56.48	59.38	58.97	58.05	59.57
192	372	8	0.1	16	0.00100	36	0.000000e+00	0.000000e+00	56.46	60.72	56.22	64.08	64.95	58.67	52.60	58.12	56.16	58.84	63.71	63.64	71.15	61.48	60.76	62.32	62.83	59.74	59.00	59.25	64.38	60.63	61.50	60.78	60.09
195	372	8	0.1	16	0.00100	36	0.000000e+00	1.000000e-08	86.10	56.16	58.91	60.65	56.99	54.28	87.14	57.53	58.88	58.31	55.89	54.51	87.94	57.90	57.58	90.10	50.19	59.03	57.39	55.41	62.10	61.45	58.31	58.28	56.47
204	372	8	0.1	16	0.00100	36	1.000000e-08	0.000000e+00	67.38	56.58	66.44	64.31	63.48	66.39	43.72	62.81	64.61	65.92	56.96	43.72	56.96	63.78	60.90	43.72	64.55	65.03	59.62	58.64	67.02	62.37	60.67	65.48	56.75
207	372	8	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	67.68	68.79	68.78	67.94	67.85	66.43	69.18	64.75	66.58	64.38	67.38	63.50	67.26	64.36	66.40	71.82	69.49	66.18	65.72	63.18	64.90	71.23	67.92	68.07	68.20
256	372	8	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	86.20	64.05	87.63	62.79	63.92	60.11	88.10	62.45	64.07	87.34	60.39	57.83	86.95	61.27	64.63	92.24	59.32	56.39	59.77	92.46	59.18	60.61	60.90	57.16	59.91
259	372	8	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	64.49	63.22	61.79	61.09	62.59	89.97	87.92	63.02	89.38	62.12	91.86	58.58	61.06	55.48	62.05	46.71	62.76	59.58	59.86	89.58	63.15	62.28	60.32	50.80	59.49
268	372	8	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	56.95	59.12	55.68	60.01	87.42	59.49	43.72	60.77	60.78	62.80	91.23	56.31	90.20	60.30	59.52	61.62	60.90	61.31	58.27	56.09	62.42	61.54	58.42	59.90	58.64
271	372	8	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	84.02	60.94	62.61	87.00	61.58	90.58	43.72	58.39	62.26	89.60	57.26	58.79	90.10	58.56	57.35	90.17	61.33	59.33	57.43	57.88	60.94	60.34	60.88	60.26	58.89
320	372	8	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	87.94	64.65	62.17	61.04	62.46	57.04	86.61	61.29	90.47	61.19	59.23	43.72	91.18	59.52	61.15	90.43	59.74	91.46	60.84	58.32	59.05	63.77	60.16	60.63	57.86
323	372	8	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	88.35	61.23	86.23	60.61	59.33	89.55	90.24	59.74	60.41	60.76	59.04	56.87	88.98	58.23	58.85	59.00	59.01	90.92	93.15	56.00	58.99	57.60	59.09	60.54	60.10
332	372	8	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	63.57	62.59	62.52	62.35	54.84	59.18	89.30	87.38	81.80	59.19	59.46	56.16	87.11	59.22	60.06	61.12	59.85	92.87	56.58	58.80	57.34	60.97	59.20	57.73	59.34
335	372	8	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	88.53	59.96	60.22	61.27	89.95	58.07	90.72	60.70	90.41	90.37	58.59	60.34	82.30	57.75	59.59	91.46	60.03	55.97	57.65	55.78	59.52	61.48	55.21	58.71	59.36
384	372	8	0.1	16	0.00100	48	0.000000e+00	0.000000e+00	43.72	63.22	59.14	60.60	61.19	59.62	86.75	60.39	61.71	58.56	59.09	60.05	61.16	51.61	58.25	61.14	63.22	57.58	49.78	58.72	59.41	61.13	62.95	58.46	62.86
387	372	8	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	68.43	66.34	57.79	62.97	56.31	43.72	79.09	78.38	63.47	65.84	61.40	59.78	89.08	60.16	63.19	53.95	63.36	64.43	57.12	61.72	63.84	61.40	64.75	67.49	60.03
396	372	8	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	67.38	63.05	66.69	66.94	65.87	59.19	56.12	63.72	66.38	64.80	57.94	65.64	59.34	86.56	61.13	65.46	64.13	66.36	60.93	66.19	65.04	64.76	62.84	67.44	64.85
399	372	8	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	82.80	62.61	60.46	53.56	60.69	82.65	54.53	53.15	58.46	58.23	56.55	55.38	87.31	59.93	60.56	59.94	58.80	89.28	56.05	55.28	59.79	59.24	59.36	60.37	60.86
448	372	8	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	55.57	61.82	62.30	59.31	61.50	58.01	85.25	60.41	61.05	89.74	90.62	58.29	89.17	62.83	61.85	89.23	61.40	61.19	58.62	58.62	57.31	61.72	61.60	62.20	57.41
451	372	8	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	88.14	62.84	61.56	60.21	88.38	58.81	89.35	88.82	61.80	61.23	59.60	58.25	89.41	59.19	60.99	55.61	59.68	89.98	58.51	87.52	63.16	61.19	60.15	58.90	60.15
460	372	8	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	62.67	62.13	58.49	63.52	60.76	58.72	87.94	61.25	60.83	61.78	56.54	88.06	88.60	60.72	60.79	55.88	61.57	59.62	60.02	58.71	62.10	58.56	62.14	58.83	59.03
463	372	8	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	89.13	62.47	61.20	62.03	62.08	58.88	87.86	61.97	89.74	62.83	60.90	55.49	43.72	59.43	61.91	91.94	63.77	60.01	57.97	56.71	61.91	57.70	88.09	61.04	60.43
512	372	8	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	88.05	60.61	62.77	61.26	62.76	90.72	62.77	59.57	62.33	61.92	58.92	57.92	43.72	59.72	60.89	81.65	60.93	59.81	59.16	60.36	59.04	60.92	61.19	58.63	58.74
515	372	8	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	64.15	62.50	56.39	61.62	61.73	91.16	89.60	61.43	91.12	60.66	58.01	55.53	86.05	59.33	61.17	84.83	63.09	60.52	59.68	59.13	59.96	60.53	57.69	57.72	59.07
524	372	8	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	85.94	62.43	61.67	61.85	61.53	88.70	88.42	60.99	90.72	61.07	91.47	56.56	61.30	57.87	60.54	54.93	61.29	91.22	56.56	55.39	57.49	59.57	59.60	58.26	59.35
527	372	8	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	87.89	62.31	62.06	62.19	62.80	59.24	88.41	61.47	91.55	85.14	60.77	59.33	90.96	60.50	60.55	88.10	58.87	60.20	58.57	58.00	88.60	58.49	59.77	59.62	56.70
576	372	8	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	67.51	66.81	66.12	66.51	74.73	61.66	57.16	67.62	69.41	68.02	62.68	66.25	70.24	65.14	65.76	63.59	68.62	65.56	61.68	62.39	67.30	66.41	69.03	66.25	67.93
579	372	8	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	58.28	62.01	65.54	65.76	61.46	60.81	64.73	61.38	59.04	64.41	59.11	59.78	62.34	59.18	64.06	64.62	64.73	61.73	60.59	59.86	66.04	64.22	64.29	67.28	62.98
588	372	8	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	85.53	62.01	64.33	63.83	85.90	60.27	65.88	62.63	63.15	63.80	60.50	61.18	62.77	61.20	65.79	60.62	60.07	63.47	60.67	58.51	63.57	64.35	61.92	64.46	62.49
591	372	8	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	87.51	61.13	63.74	65.85	66.13	43.72	57.99	62.94	65.40	65.93	63.28	59.34	63.15	59.53	61.07	63.72	56.07	65.10	61.87	59.96	64.96	43.72	43.72	61.59	61.56
640	372	8	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	86.78	62.69	64.46	62.29	88.23	59.84	60.84	59.91	63.52	90.09	91.73	57.12	91.82	60.06	59.48	60.73	60.73	63.46	57.44	58.95	63.10	62.49	59.24	63.68	62.98
643	372	8	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	63.25	64.65	64.14	64.25	64.17	89.06	88.04	61.65	43.72	63.50	58.71	60.17	63.52	62.82	62.86	88.07	60.38	84.08	60.33	58.54	62.77	64.09	60.70	59.95	62.66
652	372	8	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	86.60	64.23	61.25	61.42	60.72	90.32	86.21	59.42	88.00	61.24	90.18	59.28	90.28	59.43	62.93	60.08	61.45	58.34	59.40	57.95	59.84	62.49	58.55	60.35	61.88
655	372	8	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	60.87	61.66	63.98	60.98	63.00	43.72	63.59	58.88	61.96	62.80	59.34	59.57	87.64	60.83	61.28	60.33	62.91	64.17	56.40	58.96	62.21	62.26	62.14	60.77	60.87
704	372	8	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	65.16	62.53	62.79	88.89	63.26	59.18	66.61	61.00	63.52	64.26	90.78	60.05	88.79	61.57	61.30	91.71	60.66	90.47	57.54	91.96	65.10	58.98	59.80	58.59	65.50
707	372	8	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	86.31	62.85	61.44	61.59	86.99	59.39	87.94	60.53	56.35	61.07	59.00	60.06	90.95	59.56	60.84	91.88	60.70	60.23	58.62	58.07	60.33	58.11	59.19	57.84	59.97
716	372	8	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	88.46	62.56	63.61	43.72	88.97	90.42	87.78	60.78	84.99	89.48	60.39	57.76	89.44	57.22	59.08	85.94	59.34	60.60	57.05	92.21	87.43	58.21	58.15	60.01	59.02
719	372	8	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	63.03	87.05	56.19	62.66	88.12	58.30	90.22	59.83	61.53	60.41	55.15	59.01	90.45	58.68	63.08	60.81	59.99	91.11	60.77	89.10	58.98	58.44	61.12	60.42	62.49
768	372	8	0.1	16	0.00010	36	0.000000e+00	0.000000e+00	60.16	61.77	62.59	61.36	54.63	58.52	53.54	43.72	63.77	58.51	57.27	58.21	43.72	60.19	58.07	84.57	59.82	58.65	59.72	56.34	63.00	61.61	62.77	57.33	58.19
771	372	8	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	64.96	61.40	61.01	64.98	64.66	59.58	63.07	62.66	64.15	61.40	60.84	59.65	87.68	62.64	62.34	63.41	62.27	61.58	57.99	59.19	64.23	63.72	62.18	66.20	66.17
780	372	8	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	65.52	65.53	64.55	61.45	64.51	59.20	54.21	64.36	64.08	66.78	62.93	58.25	61.61	61.76	61.80	62.59	61.05	63.72	59.31	59.69	62.18	63.18	63.13	62.75	62.09
783	372	8	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	66.41	64.26	63.32	64.49	64.73	61.80	66.34	64.61	64.06	62.51	59.15	59.03	64.50	62.32	63.75	67.05	65.42	63.31	59.42	57.43	67.05	63.80	67.74	65.61	67.35
832	372	8	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	63.11	64.16	88.09	62.34	61.70	62.14	82.78	62.95	49.83	63.49	87.38	61.40	63.37	61.57	62.46	63.79	61.97	61.47	60.05	90.75	64.82	61.42	61.32	59.75	61.13
835	372	8	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	87.47	62.41	89.51	89.18	63.39	61.01	89.56	83.26	86.89	61.55	91.82	56.69	90.18	60.20	58.71	61.22	59.00	58.02	56.86	58.98	60.75	58.11	60.39	59.49	57.66
844	372	8	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	55.50	60.34	55.93	87.84	60.00	58.14	87.07	59.65	62.18	60.22	57.67	58.22	89.47	59.74	60.94	88.30	50.41	60.28	58.39	57.82	62.07	59.86	60.17	60.76	62.66
847	372	8	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	88.46	86.61	61.60	60.40	63.10	59.16	85.88	60.11	55.63	62.34	90.63	58.39	90.63	61.86	60.48	62.45	60.25	63.86	91.04	58.40	61.51	59.93	60.77	61.60	59.38
896	372	8	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	88.57	61.95	61.41	63.75	86.41	85.71	89.53	60.79	43.72	83.37	59.06	59.69	88.95	58.75	62.96	90.56	59.49	61.76	57.81	56.14	59.05	62.53	58.91	59.20	61.40
899	372	8	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	85.92	61.84	89.24	88.21	61.23	57.30	88.03	61.34	60.66	60.41	56.87	57.80	91.44	59.93	61.84	89.73	57.28	91.76	58.30	93.22	58.34	57.87	60.68	59.20	58.16
908	372	8	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	90.32	89.64	64.51	60.15	88.21	91.68	84.95	59.55	61.58	89.38	87.44	88.52	89.54	57.91	60.00	91.01	57.56	61.01	83.96	54.85	59.92	58.83	59.53	58.03	56.89
911	372	8	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	87.15	61.02	89.21	89.40	89.51	59.47	90.29	61.72	89.72	61.05	91.42	92.42	90.91	91.60	59.14	92.11	61.52	92.24	57.28	59.07	62.80	60.77	58.26	49.09	59.96
960	372	8	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	65.91	64.19	56.86	63.70	60.83	64.22	64.98	62.42	63.36	54.32	55.39	63.42	64.12	63.08	60.96	63.25	60.98	64.39	58.98	63.46	67.42	64.95	63.46	66.85	65.81
963	372	8	0.1	16	0.00010	48	0.000000e+00	1.000000e-08	60.69	63.38	63.31	60.97	62.14	58.30	65.03	60.89	62.25	61.48	59.71	56.45	62.62	60.54	62.69	62.77	61.05	64.40	56.54	55.20	63.71	61.08	56.79	62.77	58.85
972	372	8	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	65.10	67.66	68.03	66.09	64.82	62.07	65.64	65.81	65.91	64.17	60.81	60.65	54.60	64.27	65.55	64.51	65.68	67.47	60.60	61.30	64.23	63.97	60.88	64.32	62.25
975	372	8	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	63.21	62.80	61.77	64.95	65.57	59.69	55.52	60.69	64.89	61.67	60.72	58.76	58.10	60.20	61.52	64.37	62.84	62.43	58.23	57.43	64.31	62.42	62.48	62.50	66.99
1024	372	8	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	64.28	61.86	71.58	61.55	63.15	59.53	88.43	60.36	60.28	62.11	59.09	56.64	78.46	89.16	63.99	77.15	59.31	89.65	60.28	56.91	66.13	64.23	59.63	58.79	61.52
1027	372	8	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	64.68	61.28	86.75	59.92	65.17	90.99	87.70	59.97	61.21	43.72	89.96	91.36	61.07	85.39	58.60	56.47	60.74	58.89	90.94	56.03	61.17	60.22	61.84	54.98	61.23
1036	372	8	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	78.76	61.66	59.88	59.80	60.57	56.68	86.29	61.12	43.72	53.56	54.76	58.11	58.89	58.31	59.37	91.86	54.81	91.09	56.92	57.51	58.45	61.72	60.11	59.77	57.38
1039	372	8	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	55.62	63.42	87.85	61.10	87.49	59.55	62.84	58.74	61.71	62.20	58.10	57.57	62.31	61.98	58.59	88.74	60.54	86.85	91.83	57.09	81.30	62.08	59.01	60.09	59.99
1088	372	8	0.1	64	0.00010	48	0.000000e+00	0.000000e+00	89.21	61.29	88.37	61.99	62.29	59.43	56.03	60.52	90.47	43.72	59.18	58.93	89.93	91.33	43.72	62.26	43.72	60.94	59.01	58.01	57.96	60.18	59.41	60.52	60.11
1091	372	8	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	88.33	60.52	89.77	61.74	61.74	58.55	61.49	60.21	59.13	90.22	59.93	57.09	91.08	59.88	60.86	91.96	59.19	58.68	55.27	55.86	62.23	63.99	59.63	50.32	59.19
1100	372	8	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	87.45	64.13	62.34	62.60	63.56	91.34	89.45	62.28	60.89	60.11	59.25	59.21	92.45	60.30	61.26	89.90	60.64	60.42	92.11	86.69	59.35	59.96	56.95	59.66	49.04
1103	372	8	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	63.03	89.71	63.72	61.20	62.54	91.95	87.41	60.60	61.47	89.72	55.37	60.25	88.48	60.93	60.43	90.67	59.64	59.23	93.43	58.25	56.40	58.01	58.74	59.47	59.23
1152	372	8	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	68.04	65.12	62.78	62.22	66.18	59.08	54.79	65.32	67.11	66.64	61.77	59.17	65.81	69.12	63.05	58.04	64.07	65.58	48.36	60.58	65.49	65.53	67.19	65.33	65.35
1155	372	8	0.1	16	0.00001	24	0.000000e+00	1.000000e-08	83.84	58.72	60.58	60.97	61.02	85.43	55.48	59.37	54.47	88.76	61.50	57.39	58.91	58.22	57.21	45.27	58.32	55.76	57.88	79.73	63.01	66.43	59.82	56.64	62.51
1164	372	8	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	68.03	61.94	66.68	63.39	64.62	62.24	56.91	64.55	66.55	64.90	61.15	60.15	65.30	62.63	65.09	64.77	65.28	80.75	60.78	58.40	62.09	64.49	65.55	68.00	68.40
1167	372	8	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	64.64	62.97	56.50	61.78	61.13	62.02	65.05	61.36	63.12	64.09	60.48	61.79	87.10	60.02	65.79	63.95	61.57	62.27	59.01	59.03	64.07	62.37	63.88	63.00	62.98
1216	372	8	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	63.27	61.16	63.40	63.89	63.04	59.41	87.71	61.60	60.34	60.50	88.77	57.82	90.37	60.21	61.32	61.61	62.34	62.51	83.91	88.91	64.41	61.35	62.58	66.01	62.19
1219	372	8	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	88.66	88.44	62.31	61.67	60.50	90.84	87.20	63.42	89.36	62.82	86.75	90.70	56.03	60.84	60.66	84.42	59.35	61.33	59.03	59.83	63.82	61.67	60.31	63.62	59.40
1228	372	8	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	55.22	62.51	57.19	62.87	86.02	90.21	61.55	63.38	62.06	68.35	57.20	60.12	69.76	60.10	61.73	89.82	62.26	58.29	56.28	57.34	60.79	62.32	60.43	58.66	60.60
1231	372	8	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	88.83	62.54	62.13	61.70	60.89	91.50	62.33	60.16	62.73	62.42	91.21	60.65	92.54	83.02	62.51	91.30	60.34	61.52	60.32	57.10	60.30	62.47	62.05	61.14	60.31
1280	372	8	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	89.24	60.81	61.86	90.37	88.42	91.56	88.44	60.67	61.58	62.84	61.71	59.01	90.91	59.66	59.41	91.21	58.11	59.22	57.80	58.32	60.55	56.03	60.52	59.43	60.06
1283	372	8	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	55.01	62.48	60.66	60.22	86.25	57.11	90.22	60.94	83.22	59.48	56.40	87.88	91.58	57.84	60.40	59.84	59.09	57.22	58.02	57.54	57.78	55.85	56.04	58.01	57.99
1292	372	8	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	88.20	61.53	88.59	60.81	60.97	85.56	61.04	59.29	90.47	90.77	86.43	57.82	83.30	58.59	58.88	58.46	57.77	90.28	87.58	56.33	58.26	57.19	55.73	57.33	57.50
1295	372	8	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	62.34	63.77	61.30	60.57	89.71	90.78	82.91	60.25	88.36	61.22	59.57	58.89	90.76	57.74	60.37	90.94	60.71	62.41	58.86	58.52	61.65	63.82	50.42	59.06	56.62
1344	372	8	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	71.39	70.84	68.74	72.69	65.30	66.23	52.13	70.63	69.59	69.62	63.59	62.70	67.55	66.27	66.65	53.07	69.81	67.78	43.72	65.17	70.13	69.81	66.47	66.03	70.97
1347	372	8	0.1	16	0.00001	36	0.000000e+00	1.000000e-08	58.03	63.17	61.47	63.16	63.09	60.96	56.91	60.62	64.26	63.49	62.16	58.47	88.36	61.63	62.15	57.33	58.90	59.63	60.51	57.81	63.55	60.41	53.26	66.94	62.28
1356	372	8	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	65.97	66.98	63.13	64.24	63.31	61.58	86.68	55.07	66.03	64.81	61.48	60.32	63.91	63.17	64.95	63.63	63.87	64.10	60.55	60.26	69.01	67.68	63.21	66.64	63.31
1359	372	8	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	65.93	63.97	62.80	64.96	65.91	56.07	55.57	65.26	87.18	64.97	60.02	60.39	56.84	64.63	63.03	63.34	63.20	64.10	61.48	59.69	66.77	68.82	62.23	64.56	64.68
1408	372	8	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	62.90	61.88	87.63	62.35	58.77	88.62	56.75	60.89	59.87	59.27	59.34	57.65	88.67	56.66	59.29	60.56	59.38	58.28	58.53	55.63	62.14	63.61	61.14	59.60	59.63
1411	372	8	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	60.94	55.95	62.75	58.21	88.28	58.27	88.97	59.57	57.61	60.83	90.78	57.05	55.88	57.14	58.43	88.13	55.69	90.53	59.27	57.86	61.78	60.87	63.52	59.12	60.03
1420	372	8	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	57.91	63.17	64.46	64.01	63.21	90.04	66.62	65.11	62.63	61.18	59.39	60.85	89.39	90.74	62.02	60.22	62.09	62.39	59.10	58.94	61.44	61.37	61.12	68.22	59.79
1423	372	8	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	63.53	62.26	62.03	88.78	60.23	61.44	64.88	60.90	62.50	63.05	89.47	60.47	91.03	61.18	61.54	61.97	61.07	60.87	90.18	58.54	62.23	61.35	62.99	60.80	57.90
1472	372	8	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	88.47	61.01	86.16	61.49	62.09	60.60	86.46	60.27	90.22	62.21	59.92	56.31	92.51	59.82	59.89	58.92	59.55	91.67	58.66	90.47	64.71	55.74	59.63	61.16	59.37
1475	372	8	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	87.97	61.39	87.45	60.96	60.47	59.45	62.04	59.51	61.07	88.99	90.81	59.30	91.43	60.36	60.28	58.45	60.05	58.41	57.83	56.59	57.89	59.67	56.56	57.91	59.79
1484	372	8	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	90.19	61.56	61.79	61.22	62.60	56.78	50.38	60.90	62.77	90.04	91.37	60.04	89.72	59.84	58.61	60.06	58.93	59.32	89.66	57.92	62.67	60.60	58.53	58.04	58.91
1487	372	8	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	87.47	64.43	89.80	62.19	89.22	58.85	88.69	59.06	90.67	62.82	92.31	60.10	60.73	60.69	60.02	45.81	61.11	90.60	59.90	56.88	89.43	58.11	57.67	58.24	58.60
1536	372	8	0.1	16	0.00001	48	0.000000e+00	0.000000e+00	45.90	62.56	63.05	64.01	62.10	61.12	49.74	62.39	65.03	64.89	62.12	43.72	60.47	61.61	54.73	63.19	65.29	54.90	59.28	56.56	68.98	64.81	65.18	66.11	62.32
1539	372	8	0.1	16	0.00001	48	0.000000e+00	1.000000e-08	56.64	59.26	64.68	60.21	56.30	58.48	85.71	57.72	54.84	62.56	60.94	55.98	87.48	59.37	63.68	58.30	64.03	62.16	57.06	56.53	62.60	61.17	62.23	61.77	58.32
1548	372	8	0.1	16	0.00001	48	1.000000e-08	0.000000e+00	64.36	63.72	65.04	62.34	66.80	56.95	68.39	61.14	57.16	86.27	54.80	61.08	63.35	62.98	60.26	60.19	59.96	55.50	58.79	56.44	62.83	60.23	62.79	66.89	60.24
1551	372	8	0.1	16	0.00001	48	1.000000e-08	1.000000e-08	63.86	63.61	65.06	64.44	65.30	57.81	57.61	63.63	62.57	63.62	56.11	59.44	58.04	62.13	63.22	55.54	60.51	64.70	58.95	61.29	65.80	65.78	64.75	67.60	59.14

Test Data mAP Results Preview:

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_zero	mAP_test_zero_2	mAP_test_zero_3	mAP_test_zero_4	mAP_test_zero_5	mAP_test_zero_6	mAP_test_zero_7	mAP_test_zero_8	mAP_test_zero_9	mAP_test_zero_10	mAP_test_zero_11	mAP_test_zero_12	mAP_test_zero_13	mAP_test_zero_14	mAP_test_zero_15	mAP_test_zero_16	mAP_test_zero_17	mAP_test_zero_18	mAP_test_zero_19	mAP_test_zero_20	mAP_test_zero_21	mAP_test_zero_22	mAP_test_zero_23	mAP_test_zero_25	mAP_test_zero_26
0	372	8	0.1	16	0.00100	24	0.000000e+00	0.000000e+00	60.67	60.61	60.96	60.07	53.06	58.94	59.40	59.31	63.04	60.85	60.50	58.46	64.07	63.09	60.37	62.88	61.42	59.96	59.06	59.27	61.20	62.72	62.66	61.76	59.99
3	372	8	0.1	16	0.00100	24	0.000000e+00	1.000000e-08	49.82	58.05	59.20	59.52	59.30	60.40	61.02	60.05	60.38	60.51	58.24	58.40	61.22	49.91	59.66	48.12	59.87	62.29	60.77	57.58	61.30	61.74	63.03	61.17	62.83
12	372	8	0.1	16	0.00100	24	1.000000e-08	0.000000e+00	50.55	48.12	60.45	58.59	58.15	60.20	48.55	60.57	60.72	51.90	48.12	60.85	60.53	60.21	59.11	60.74	61.01	57.99	60.80	57.36	62.23	48.12	61.35	60.78	62.39
15	372	8	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	60.25	58.62	60.35	60.99	59.26	60.33	49.61	48.88	60.28	60.67	60.22	58.65	61.05	58.88	59.94	59.17	60.63	59.96	59.96	59.63	61.82	61.43	61.69	61.15	60.55
64	372	8	0.1	32	0.00100	24	0.000000e+00	0.000000e+00	51.78	48.12	57.82	61.53	61.00	49.35	58.06	59.80	60.24	59.88	60.02	61.65	59.18	49.70	58.85	60.74	60.11	61.06	58.25	58.00	60.52	61.02	62.15	62.04	62.87
67	372	8	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	56.80	61.84	57.16	61.15	61.08	48.12	63.56	60.06	58.39	60.68	55.46	60.97	60.51	60.13	59.78	57.69	59.59	57.67	58.96	59.85	61.95	61.83	62.96	62.88	61.44
76	372	8	0.1	32	0.00100	24	1.000000e-08	0.000000e+00	60.97	59.58	61.08	58.04	60.60	59.52	57.26	59.52	48.12	56.81	62.29	50.54	86.80	60.97	60.80	57.42	60.06	59.56	49.26	59.32	61.27	62.83	63.37	61.39	61.24
79	372	8	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	59.05	62.10	55.96	60.60	60.28	59.32	59.52	60.16	57.15	60.28	59.88	62.70	87.16	60.08	48.12	60.37	59.74	60.08	60.77	61.29	60.79	62.25	62.69	62.20	61.63
128	372	8	0.1	64	0.00100	24	0.000000e+00	0.000000e+00	61.88	62.45	60.22	61.65	60.20	57.74	60.39	61.90	59.22	61.99	61.31	60.15	58.05	60.85	60.96	59.43	61.17	60.93	60.82	60.90	61.66	61.72	62.84	62.19	62.60
131	372	8	0.1	64	0.00100	24	0.000000e+00	1.000000e-08	59.61	62.50	58.24	60.50	60.37	60.86	57.79	51.39	50.06	50.00	60.77	61.65	59.63	61.85	61.23	60.85	60.72	57.88	61.31	51.06	62.44	61.83	62.34	62.18	62.40
140	372	8	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	59.00	61.40	61.28	61.15	59.77	59.80	59.23	60.44	60.77	82.94	49.80	60.17	59.73	61.43	61.99	48.32	58.81	61.99	61.25	61.15	61.60	61.82	59.63	61.39	62.45
143	372	8	0.1	64	0.00100	24	1.000000e-08	1.000000e-08	62.47	61.82	61.81	59.12	61.64	62.38	58.41	60.93	60.67	61.46	59.77	61.99	58.89	60.88	60.88	50.37	60.78	58.54	48.12	61.93	62.24	62.34	61.89	62.37	62.13
192	372	8	0.1	16	0.00100	36	0.000000e+00	0.000000e+00	48.96	60.53	49.93	60.50	60.36	58.40	55.12	61.58	49.67	61.16	57.98	58.14	64.76	60.40	59.24	61.34	60.11	59.90	59.56	57.66	60.99	63.39	62.09	60.92	59.59
195	372	8	0.1	16	0.00100	36	0.000000e+00	1.000000e-08	57.87	58.88	59.12	58.02	56.72	58.32	56.48	58.58	58.66	58.69	59.44	58.75	57.53	57.53	58.69	58.17	55.62	59.63	58.74	58.64	61.61	62.23	60.60	60.71	61.79
204	372	8	0.1	16	0.00100	36	1.000000e-08	0.000000e+00	60.32	48.61	60.64	59.38	61.07	60.99	48.12	60.07	59.16	60.48	50.65	48.12	49.65	57.21	61.02	48.12	60.65	59.66	59.73	58.80	60.99	64.15	62.29	61.66	60.84
207	372	8	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	60.28	60.98	59.21	60.77	60.90	60.15	59.43	61.04	60.33	60.30	60.48	60.70	60.91	60.93	60.23	57.42	58.95	60.43	60.18	59.77	61.30	59.53	60.69	61.06	60.09
256	372	8	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	57.17	61.06	58.92	61.99	60.47	60.84	57.91	62.25	61.19	58.02	62.79	59.37	57.89	60.58	60.81	59.02	60.95	50.61	60.48	59.35	62.25	62.78	62.95	63.48	62.10
259	372	8	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	61.16	60.37	61.37	60.34	61.22	59.25	57.47	61.44	57.71	59.51	57.43	61.43	61.23	49.19	60.82	75.41	59.66	61.18	60.09	59.57	61.78	62.14	61.94	45.25	61.80
268	372	8	0.1	32	0.00100	36	1.000000e-08	0.000000e+00	49.88	60.93	50.29	59.93	59.04	59.78	48.12	61.40	61.53	60.03	57.60	59.74	58.83	60.48	60.79	59.38	61.01	60.59	59.97	58.16	61.25	60.41	61.74	62.05	60.95
271	372	8	0.1	32	0.00100	36	1.000000e-08	1.000000e-08	56.15	60.80	61.67	59.02	59.42	57.73	48.12	59.83	61.02	58.63	59.33	61.14	57.35	60.68	60.34	57.59	60.75	62.30	59.20	61.25	62.19	62.17	61.60	61.72	60.56
320	372	8	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	57.74	61.29	61.62	61.93	60.77	60.06	56.64	61.70	58.40	61.11	61.19	48.12	58.59	60.72	60.79	58.82	60.57	58.51	61.65	61.36	61.91	61.68	62.26	62.51	62.12
323	372	8	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	57.83	61.81	59.17	61.87	60.13	56.31	58.72	61.65	60.51	60.54	61.59	61.65	59.34	60.10	61.59	59.49	60.88	57.91	58.61	60.58	61.90	62.05	61.66	61.73	61.84
332	372	8	0.1	64	0.00100	36	1.000000e-08	0.000000e+00	61.32	61.59	61.29	61.59	50.26	62.32	58.89	59.82	91.95	60.55	60.77	59.56	50.67	61.94	60.77	60.50	61.93	58.51	60.77	61.98	62.43	62.48	61.53	62.87	62.68
335	372	8	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	57.96	61.45	61.17	61.87	59.53	60.65	58.88	62.13	58.08	58.25	61.59	61.25	85.74	61.29	60.83	59.02	60.02	50.06	60.74	59.62	61.72	61.51	63.37	61.99	61.93
384	372	8	0.1	16	0.00100	48	0.000000e+00	0.000000e+00	48.12	58.36	60.24	60.15	58.92	59.13	57.33	59.19	59.99	59.21	59.21	59.24	60.57	53.04	58.85	60.43	60.52	62.17	56.33	58.98	62.79	61.98	62.01	63.69	61.68
387	372	8	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	60.28	60.99	49.50	59.37	50.61	48.12	49.46	35.10	60.22	60.64	59.80	59.42	57.88	58.88	60.57	62.76	59.64	59.94	58.54	59.67	61.56	60.81	60.91	60.39	62.57
396	372	8	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	59.09	60.25	61.54	60.60	59.90	48.56	50.43	59.49	59.00	59.19	58.53	59.02	46.16	59.23	59.25	58.64	60.31	60.21	58.66	58.28	61.54	62.59	61.58	60.78	61.88
399	372	8	0.1	16	0.00100	48	1.000000e-08	1.000000e-08	55.81	58.75	58.97	50.26	61.71	48.84	48.07	51.67	57.46	58.61	59.64	59.73	57.37	60.88	60.15	59.20	58.42	56.23	50.08	58.75	60.85	59.92	61.82	60.96	61.24
448	372	8	0.1	32	0.00100	48	0.000000e+00	0.000000e+00	49.74	60.70	59.27	59.49	60.95	58.21	57.43	61.39	59.76	58.22	57.52	60.47	57.69	60.39	60.02	57.40	60.75	61.28	60.48	59.54	62.63	60.89	61.77	61.92	61.06
451	372	8	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	57.49	60.41	60.37	61.26	57.29	61.01	59.19	59.88	60.64	60.63	60.94	60.24	57.44	60.33	60.83	50.08	60.34	57.60	59.32	51.34	61.52	59.82	61.81	61.51	62.18
460	372	8	0.1	32	0.00100	48	1.000000e-08	0.000000e+00	61.24	60.14	67.49	62.06	61.15	60.17	59.28	60.46	60.77	61.35	60.08	60.33	58.07	60.76	60.17	49.51	59.84	58.92	61.99	60.67	61.79	60.20	62.55	62.57	61.80
463	372	8	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	57.99	60.64	60.62	62.36	60.89	58.51	57.33	60.08	58.88	60.71	61.62	58.61	48.12	59.93	60.71	59.52	61.02	60.08	61.09	59.38	62.26	62.76	60.03	62.28	61.94
512	372	8	0.1	64	0.00100	48	0.000000e+00	0.000000e+00	57.84	58.74	61.53	61.64	61.40	58.30	61.26	61.53	59.63	60.74	61.59	61.18	48.12	60.75	60.93	86.47	60.97	61.11	61.31	61.41	63.02	61.69	61.39	62.30	61.43
515	372	8	0.1	64	0.00100	48	0.000000e+00	1.000000e-08	61.58	61.66	49.59	61.66	61.40	60.22	57.99	61.70	58.21	60.99	62.70	50.25	50.77	61.16	62.00	50.56	60.63	61.39	61.37	59.62	61.63	61.86	62.67	63.04	62.29
524	372	8	0.1	64	0.00100	48	1.000000e-08	0.000000e+00	56.61	60.61	61.38	61.71	61.18	56.17	58.35	61.65	59.67	61.29	58.86	59.92	61.45	61.66	61.36	77.89	60.82	58.98	59.67	59.66	62.40	62.16	61.83	61.65	60.93
527	372	8	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	59.57	61.65	62.06	61.97	61.93	60.21	57.26	61.17	59.84	90.67	61.93	61.84	59.44	62.08	61.88	50.58	60.67	62.08	61.56	60.65	60.46	62.85	62.18	62.57	62.59
576	372	8	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	60.44	61.24	59.49	60.16	61.74	60.65	47.45	59.84	59.88	59.75	59.42	59.23	59.25	65.29	60.63	57.96	59.14	58.87	59.39	59.35	61.48	60.57	60.60	61.10	60.83
579	372	8	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	48.60	60.56	59.58	61.16	59.34	60.09	59.36	60.90	49.45	61.40	60.48	59.54	61.29	59.47	59.97	58.57	60.46	59.23	60.17	59.31	60.95	61.78	60.88	60.74	61.93
588	372	8	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	56.41	62.26	59.97	59.18	58.01	59.13	60.52	59.74	58.56	59.96	59.80	61.50	59.41	59.37	60.85	60.52	59.27	59.80	60.03	58.47	60.93	62.67	62.80	62.53	62.48
591	372	8	0.1	16	0.00010	24	1.000000e-08	1.000000e-08	59.50	59.41	59.55	60.24	59.07	48.12	48.41	59.04	60.19	59.17	61.13	58.81	61.37	59.84	58.91	61.47	50.41	59.79	60.61	59.42	61.88	48.12	48.12	62.39	62.19
640	372	8	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	57.99	60.21	59.58	61.01	57.73	60.32	58.03	59.93	61.73	58.42	59.64	58.67	58.51	60.82	60.33	59.90	60.56	60.29	59.71	60.40	61.68	62.04	62.68	61.52	61.60
643	372	8	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	61.36	60.82	61.13	61.09	61.63	59.74	58.58	61.39	48.12	60.83	60.42	60.58	60.42	60.41	59.95	57.77	60.18	82.77	60.06	60.17	60.98	61.57	61.66	62.07	61.23
652	372	8	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	56.36	61.68	60.05	61.44	60.10	57.62	56.24	61.40	58.61	61.16	59.15	61.47	58.97	60.51	61.25	60.77	60.48	60.37	61.59	59.21	62.44	61.45	61.79	61.78	61.82
655	372	8	0.1	32	0.00010	24	1.000000e-08	1.000000e-08	60.40	62.65	61.22	62.00	61.82	48.12	61.47	60.20	61.09	60.62	61.52	61.66	57.44	60.59	61.80	60.62	60.08	59.48	59.88	61.15	61.62	61.27	60.53	62.37	61.60
704	372	8	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	61.16	61.87	61.27	59.81	61.93	60.11	59.64	61.76	60.67	61.39	60.33	62.39	48.50	60.90	61.75	59.43	61.49	59.45	62.97	59.04	60.33	62.55	62.32	62.42	61.84
707	372	8	0.1	64	0.00010	24	0.000000e+00	1.000000e-08	60.22	60.66	61.89	61.29	57.85	61.99	59.09	62.06	48.88	62.32	60.77	61.77	59.21	61.36	61.54	59.68	60.98	61.75	61.47	61.81	62.45	62.79	62.81	62.79	62.19
716	372	8	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	59.45	61.99	61.92	48.12	59.52	60.28	58.03	61.82	49.18	58.63	61.99	62.00	59.42	61.82	60.75	51.35	61.10	60.51	61.65	59.06	59.87	62.26	61.49	62.49	61.76
719	372	8	0.1	64	0.00010	24	1.000000e-08	1.000000e-08	61.05	59.74	49.49	61.03	60.12	60.38	58.25	60.54	59.85	61.46	49.75	61.99	59.20	60.18	61.51	60.74	60.85	59.52	61.64	88.98	62.50	62.36	61.50	62.00	61.18
768	372	8	0.1	16	0.00010	36	0.000000e+00	0.000000e+00	58.30	60.70	59.26	58.29	50.69	59.58	51.02	48.12	58.40	61.07	58.42	58.50	48.12	61.08	60.50	59.86	66.09	62.36	57.21	57.74	61.19	62.29	61.48	62.96	62.60
771	372	8	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	60.65	60.53	59.46	60.30	59.33	59.11	58.86	59.18	59.91	58.47	60.42	58.42	57.63	59.61	60.41	59.84	59.15	58.18	58.54	58.78	61.00	62.26	62.76	61.68	61.80
780	372	8	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	59.99	60.00	60.11	59.55	58.38	58.81	51.02	61.32	59.51	60.53	60.17	59.44	60.04	59.38	60.91	59.38	59.46	59.86	59.16	59.99	62.31	60.74	61.60	61.28	62.15
783	372	8	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	60.49	59.77	59.33	60.97	60.61	59.39	59.78	60.49	58.70	60.10	59.75	59.84	59.18	60.14	59.75	60.20	58.83	59.11	58.84	58.61	60.36	61.40	60.32	61.73	60.09
832	372	8	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	60.48	61.67	58.67	59.36	60.96	60.98	58.86	60.32	53.08	61.37	47.75	61.53	60.41	60.57	60.37	59.78	60.35	60.99	62.40	57.79	61.06	62.22	62.17	61.79	62.62
835	372	8	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	58.08	61.37	58.97	58.38	60.80	62.20	58.41	89.37	56.44	61.50	58.89	59.20	58.43	60.99	60.41	59.68	59.93	58.89	60.31	60.53	61.24	62.19	62.16	61.55	60.72
844	372	8	0.1	32	0.00010	36	1.000000e-08	0.000000e+00	49.52	60.31	50.47	59.82	60.21	59.41	47.21	61.57	61.39	60.96	58.51	59.61	57.60	61.12	60.53	55.99	57.83	61.08	61.84	60.56	61.91	60.11	62.32	61.67	61.08
847	372	8	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	58.93	57.38	60.72	58.88	60.38	60.76	56.87	61.05	48.86	61.25	59.16	60.53	58.98	60.47	60.42	59.65	59.80	61.07	58.03	60.54	62.27	60.86	60.80	61.32	60.75
896	372	8	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	58.90	61.42	61.01	62.00	59.72	90.60	57.93	61.20	48.12	49.79	61.53	60.11	57.97	61.67	61.07	59.29	62.18	61.67	61.25	59.15	61.79	62.50	62.00	62.89	62.17
899	372	8	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	55.40	61.76	59.07	59.73	61.54	59.14	59.78	62.11	62.00	60.96	59.71	61.04	59.04	61.46	62.34	56.88	61.58	58.35	61.65	58.99	60.92	62.45	62.07	62.63	62.41
908	372	8	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	57.79	59.56	61.55	60.44	58.19	59.95	51.41	62.00	61.36	59.18	51.55	51.80	60.01	61.70	62.25	59.73	61.00	61.46	88.06	59.06	61.43	61.60	61.47	62.61	62.49
911	372	8	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	59.57	62.00	59.06	59.52	59.22	61.90	59.49	60.91	59.36	62.32	60.22	59.95	59.91	58.42	62.28	59.05	60.03	59.22	62.90	60.65	61.57	62.05	62.09	48.62	62.01
960	372	8	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	60.69	58.84	48.99	60.55	60.04	59.87	61.04	58.84	60.07	50.32	51.19	60.31	59.53	60.13	59.48	59.73	62.96	58.44	59.54	59.23	61.64	62.09	62.32	67.63	61.05
963	372	8	0.1	16	0.00010	48	0.000000e+00	1.000000e-08	59.83	61.16	59.35	60.53	58.33	58.73	59.24	58.37	57.54	60.46	59.76	60.91	60.62	59.45	58.99	59.79	58.12	60.17	59.75	58.81	60.54	58.56	61.16	62.61	61.56
972	372	8	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	59.24	59.36	60.87	61.00	60.42	59.96	61.00	64.32	60.18	59.34	59.26	58.99	52.63	59.28	61.29	58.31	59.16	59.37	60.65	60.05	61.28	61.99	62.95	60.19	61.50
975	372	8	0.1	16	0.00010	48	1.000000e-08	1.000000e-08	59.52	59.23	58.49	59.84	60.71	58.79	49.29	59.67	58.27	59.23	60.22	58.57	48.01	59.49	59.33	60.23	62.34	59.70	58.66	58.88	62.07	61.43	61.84	62.19	60.56
1024	372	8	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	61.08	61.25	62.54	59.50	61.44	62.71	57.80	58.81	60.15	60.77	60.37	50.57	47.44	58.54	57.86	75.80	58.64	56.80	60.82	58.25	61.70	60.98	62.49	62.74	61.97
1027	372	8	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	60.25	59.98	58.51	60.25	60.75	59.67	58.80	59.26	58.90	48.12	50.01	59.15	59.59	47.95	60.58	49.59	60.47	60.88	58.70	59.34	60.63	62.16	60.18	62.50	61.10
1036	372	8	0.1	32	0.00010	48	1.000000e-08	0.000000e+00	87.56	60.87	60.22	60.93	58.87	58.76	50.13	58.99	48.12	64.74	51.45	60.22	59.64	61.28	60.48	57.85	50.78	56.76	60.44	60.06	61.63	62.23	61.83	62.20	60.54
1039	372	8	0.1	32	0.00010	48	1.000000e-08	1.000000e-08	50.29	61.89	57.17	60.76	57.43	61.04	60.64	60.24	60.21	60.23	59.18	60.52	59.69	60.47	61.51	57.43	60.94	85.72	58.88	60.60	85.90	62.23	62.47	61.79	61.56
1088	372	8	0.1	64	0.00010	48	0.000000e+00	0.000000e+00	58.72	60.17	58.89	61.71	61.64	61.61	49.90	61.99	58.61	48.12	61.99	61.53	58.92	60.02	48.12	61.46	48.12	61.77	61.99	61.15	62.48	62.06	62.45	62.75	61.15
1091	372	8	0.1	64	0.00010	48	0.000000e+00	1.000000e-08	58.45	59.52	58.63	60.77	62.30	60.59	61.25	61.29	61.14	58.85	62.39	61.31	58.87	60.85	61.91	58.47	60.49	61.40	50.00	59.57	60.04	61.91	60.91	50.49	62.19
1100	372	8	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	59.69	61.55	60.87	61.31	60.72	59.48	58.97	61.70	61.51	61.44	61.85	61.58	59.52	61.41	61.70	57.64	60.82	62.09	59.37	52.38	62.90	62.33	62.30	62.56	46.00
1103	372	8	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	61.88	59.48	61.29	60.75	61.99	58.62	59.27	61.15	60.92	59.12	59.17	61.99	60.19	60.59	60.40	59.33	61.36	60.98	59.95	61.87	62.46	62.69	62.56	62.08	62.24
1152	372	8	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	59.73	59.62	62.79	61.30	60.03	58.65	52.91	60.43	61.35	61.33	59.69	59.64	61.78	58.81	61.14	64.98	60.50	59.73	45.06	58.91	61.75	61.18	61.35	61.42	61.80
1155	372	8	0.1	16	0.00001	24	0.000000e+00	1.000000e-08	55.40	60.02	59.56	59.93	60.82	56.22	50.21	58.47	50.72	57.07	59.82	58.91	61.75	59.51	59.05	44.27	59.55	49.24	59.70	85.86	61.99	60.59	62.71	62.84	59.09
1164	372	8	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	60.26	59.06	60.26	59.35	59.11	59.30	49.40	59.68	58.74	58.74	60.43	59.14	61.41	61.28	59.82	59.72	60.46	59.21	60.37	57.95	62.32	59.23	62.12	60.46	60.80
1167	372	8	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	59.63	60.48	50.31	61.49	60.60	59.64	61.77	59.36	61.15	60.84	60.79	59.54	58.83	59.18	61.36	59.91	60.02	59.35	59.65	60.15	60.02	61.32	61.47	61.44	61.43
1216	372	8	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	60.29	58.88	58.32	59.47	60.96	58.90	58.26	59.61	58.79	60.37	51.35	59.47	58.42	58.62	59.09	61.54	60.81	62.28	87.22	56.55	61.69	61.39	61.60	61.30	61.64
1219	372	8	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	58.28	59.36	61.75	60.38	60.75	59.40	58.88	61.50	58.12	61.65	50.99	59.48	50.25	60.09	60.45	52.21	60.78	59.77	59.47	61.65	61.61	62.72	62.12	61.42	61.71
1228	372	8	0.1	32	0.00001	24	1.000000e-08	0.000000e+00	50.68	60.37	49.94	61.59	55.81	59.02	59.72	62.02	58.91	76.51	59.58	60.83	45.71	60.53	60.70	58.86	59.51	60.66	50.32	59.99	61.06	62.65	62.92	61.84	61.67
1231	372	8	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	58.67	60.27	60.20	61.03	59.32	60.02	61.50	61.49	59.92	61.77	59.33	61.58	59.18	86.25	59.51	59.54	60.43	61.45	61.53	59.63	63.12	61.59	62.25	61.80	62.13
1280	372	8	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	59.53	61.03	61.86	60.14	59.80	59.26	59.30	62.09	60.34	61.14	61.59	61.99	59.24	61.18	60.86	58.80	60.32	61.80	60.11	61.11	63.47	62.84	62.53	62.18	61.89
1283	372	8	0.1	64	0.00001	24	0.000000e+00	1.000000e-08	50.25	60.97	60.71	61.29	58.23	59.37	59.32	61.70	87.81	61.66	60.97	50.82	59.55	61.47	61.93	61.64	60.74	60.07	62.17	61.76	62.08	62.60	62.78	61.61	61.21
1292	372	8	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	59.39	61.99	58.51	61.33	61.43	51.74	59.62	61.93	58.40	58.62	84.92	61.70	49.68	60.50	60.58	61.39	59.85	59.62	50.60	61.87	62.20	62.14	62.58	61.96	60.89
1295	372	8	0.1	64	0.00001	24	1.000000e-08	1.000000e-08	61.10	60.69	61.93	61.81	59.71	58.51	90.46	61.70	58.97	62.08	60.64	61.82	59.23	62.39	60.97	59.27	60.07	60.14	62.64	61.68	61.42	61.55	50.01	61.84	62.16
1344	372	8	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	58.95	58.94	61.11	58.71	61.64	57.22	49.99	59.41	59.66	60.07	58.48	57.83	60.52	60.60	61.13	51.60	59.03	61.36	48.12	57.87	59.69	60.80	61.63	61.91	62.05
1347	372	8	0.1	16	0.00001	36	0.000000e+00	1.000000e-08	47.84	61.17	59.74	59.57	60.24	59.53	49.53	59.98	60.47	59.80	61.04	57.95	57.64	60.21	60.69	51.06	59.07	62.34	60.38	50.94	62.55	63.08	44.22	59.79	62.42
1356	372	8	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	59.76	60.13	60.04	60.33	60.05	59.82	57.98	50.93	60.43	60.33	62.12	59.57	59.23	59.48	60.05	60.79	59.29	58.64	60.51	59.19	60.72	60.68	61.87	60.68	61.53
1359	372	8	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	59.03	61.01	60.34	60.32	61.74	49.96	49.14	59.19	56.83	59.25	62.22	60.12	49.87	59.66	59.64	60.04	59.62	59.50	59.19	59.86	60.89	59.35	61.98	61.48	60.95
1408	372	8	0.1	32	0.00001	36	0.000000e+00	0.000000e+00	61.11	60.40	56.71	60.93	60.14	57.14	49.51	60.34	60.42	61.21	59.43	59.66	58.03	50.16	59.89	60.10	61.39	62.21	60.96	50.31	61.61	61.65	62.56	61.71	62.40
1411	372	8	0.1	32	0.00001	36	0.000000e+00	1.000000e-08	59.34	49.49	59.98	59.56	59.34	60.84	58.36	61.11	58.73	59.85	56.87	60.20	50.47	59.66	58.70	57.55	50.02	57.48	61.77	60.39	61.76	61.58	60.71	60.97	61.43
1420	372	8	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	49.86	61.27	59.20	61.63	60.02	58.28	59.78	61.16	60.09	59.62	62.13	61.93	58.32	59.24	60.30	59.66	60.67	60.68	61.03	60.44	61.77	62.32	62.36	60.26	62.34
1423	372	8	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	61.92	61.03	61.37	57.94	60.93	61.45	60.20	60.77	61.42	61.28	55.56	61.59	58.63	61.31	60.53	60.24	60.29	60.38	57.35	61.53	60.75	62.25	62.13	61.45	60.95
1472	372	8	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	58.88	62.32	56.55	60.97	61.41	60.92	88.69	61.11	57.61	60.22	62.32	59.93	58.31	62.18	61.58	60.71	60.72	58.94	60.46	50.83	60.35	49.51	62.36	62.52	61.00
1475	372	8	0.1	64	0.00001	36	0.000000e+00	1.000000e-08	59.15	61.47	59.52	61.06	61.29	61.68	61.33	61.11	61.59	59.17	60.01	62.26	59.32	62.15	61.94	61.23	61.26	59.76	61.16	61.34	62.42	61.83	62.55	61.26	61.53
1484	372	8	0.1	64	0.00001	36	1.000000e-08	0.000000e+00	59.31	59.92	61.75	60.00	61.02	49.75	51.94	61.93	61.53	59.40	57.43	61.65	59.15	61.21	60.51	58.24	60.23	60.93	52.20	60.44	61.80	61.74	62.13	61.53	62.27
1487	372	8	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	59.75	61.52	59.26	62.32	59.41	62.08	58.77	61.93	59.10	61.42	59.79	62.18	62.16	61.21	61.42	65.45	60.41	59.48	61.53	59.45	60.03	60.46	61.95	61.82	62.40
1536	372	8	0.1	16	0.00001	48	0.000000e+00	0.000000e+00	65.84	60.54	59.99	59.36	60.32	59.22	52.83	61.08	60.87	58.17	60.09	48.12	61.69	61.30	52.11	59.40	59.83	50.01	59.34	58.54	60.46	61.26	61.42	61.46	61.22
1539	372	8	0.1	16	0.00001	48	0.000000e+00	1.000000e-08	50.50	58.35	60.99	58.59	49.86	60.13	56.52	58.89	51.01	59.44	58.07	59.70	57.61	60.24	60.41	57.80	59.25	59.87	59.47	58.33	62.27	60.61	62.36	62.18	61.90
1548	372	8	0.1	16	0.00001	48	1.000000e-08	0.000000e+00	59.36	59.74	59.79	59.97	60.43	59.43	60.07	59.30	49.20	56.81	51.47	60.60	59.62	58.59	62.46	61.96	58.23	50.17	59.85	50.13	62.85	62.31	62.58	60.38	62.83
1551	372	8	0.1	16	0.00001	48	1.000000e-08	1.000000e-08	60.89	60.25	58.55	60.59	59.81	57.87	49.67	59.64	59.16	58.89	49.99	60.11	49.19	61.63	60.43	51.26	60.26	59.44	59.76	60.01	61.33	61.11	61.75	60.74	62.78

Model performance on validation data (T-test)

Statistical Significance:

Using the global mean maximum row as the reference, we perform the right-tailed t-test to identify significant hyperparameters.

pvalues = []
pvalues_real = []

for _,index in enumerate(dt_valid_sub.index):
  _, p_value = stats.ttest_ind(max_row[:-1], dt_valid_sub.loc[index][:-1], alternative='greater')
  pvalues_real.append(p_value)
  pvalues.append((p_value >= 0.05)*1)

  

dt_valid_sub['pvalues'] = pvalues_real
dt_valid_sub['sig'] = pvalues


dt_valid_sub1 = dt_valid_sub.query('sig == 1')


max_indexx = dt_valid_sub1.iloc[:, :-3].mean().idxmax()

#print(sum(pvalues),max_indexx)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_zero	mAP_valid_zero_2	mAP_valid_zero_3	mAP_valid_zero_4	mAP_valid_zero_5	mAP_valid_zero_6	mAP_valid_zero_7	mAP_valid_zero_8	mAP_valid_zero_9	mAP_valid_zero_10	mAP_valid_zero_11	mAP_valid_zero_12	mAP_valid_zero_13	mAP_valid_zero_14	mAP_valid_zero_15	mAP_valid_zero_16	mAP_valid_zero_17	mAP_valid_zero_18	mAP_valid_zero_19	mAP_valid_zero_20	mAP_valid_zero_21	mAP_valid_zero_22	mAP_valid_zero_23	mAP_valid_zero_25	mAP_valid_zero_26
1219	372	8	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	88.66	88.44	62.31	61.67	60.50	90.84	87.20	63.42	89.36	62.82	86.75	90.70	56.03	60.84	60.66	84.42	59.35	61.33	59.03	59.83	63.82	61.67	60.31	63.62	59.40
1231	372	8	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	88.83	62.54	62.13	61.70	60.89	91.50	62.33	60.16	62.73	62.42	91.21	60.65	92.54	83.02	62.51	91.30	60.34	61.52	60.32	57.10	60.30	62.47	62.05	61.14	60.31
835	372	8	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	87.47	62.41	89.51	89.18	63.39	61.01	89.56	83.26	86.89	61.55	91.82	56.69	90.18	60.20	58.71	61.22	59.00	58.02	56.86	58.98	60.75	58.11	60.39	59.49	57.66
1027	372	8	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	64.68	61.28	86.75	59.92	65.17	90.99	87.70	59.97	61.21	43.72	89.96	91.36	61.07	85.39	58.60	56.47	60.74	58.89	90.94	56.03	61.17	60.22	61.84	54.98	61.23
67	372	8	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	85.92	62.52	86.84	61.52	60.48	43.72	56.28	61.21	87.16	64.16	88.25	58.68	61.32	58.74	59.05	89.89	61.15	87.99	92.21	56.04	60.29	60.23	59.11	59.73	59.51
79	372	8	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	88.42	62.49	86.25	61.84	61.35	92.42	88.26	61.40	89.55	62.08	92.09	59.67	84.68	61.43	43.72	60.59	59.85	61.00	59.26	58.30	63.92	62.27	59.44	62.05	58.68
256	372	8	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	86.20	64.05	87.63	62.79	63.92	60.11	88.10	62.45	64.07	87.34	60.39	57.83	86.95	61.27	64.63	92.24	59.32	56.39	59.77	92.46	59.18	60.61	60.90	57.16	59.91
451	372	8	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	88.14	62.84	61.56	60.21	88.38	58.81	89.35	88.82	61.80	61.23	59.60	58.25	89.41	59.19	60.99	55.61	59.68	89.98	58.51	87.52	63.16	61.19	60.15	58.90	60.15
1280	372	8	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	89.24	60.81	61.86	90.37	88.42	91.56	88.44	60.67	61.58	62.84	61.71	59.01	90.91	59.66	59.41	91.21	58.11	59.22	57.80	58.32	60.55	56.03	60.52	59.43	60.06
1292	372	8	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	88.20	61.53	88.59	60.81	60.97	85.56	61.04	59.29	90.47	90.77	86.43	57.82	83.30	58.59	58.88	58.46	57.77	90.28	87.58	56.33	58.26	57.19	55.73	57.33	57.50
1472	372	8	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	88.47	61.01	86.16	61.49	62.09	60.60	86.46	60.27	90.22	62.21	59.92	56.31	92.51	59.82	59.89	58.92	59.55	91.67	58.66	90.47	64.71	55.74	59.63	61.16	59.37
1487	372	8	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	87.47	64.43	89.80	62.19	89.22	58.85	88.69	59.06	90.67	62.82	92.31	60.10	60.73	60.69	60.02	45.81	61.11	90.60	59.90	56.88	89.43	58.11	57.67	58.24	58.60
704	372	8	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	65.16	62.53	62.79	88.89	63.26	59.18	66.61	61.00	63.52	64.26	90.78	60.05	88.79	61.57	61.30	91.71	60.66	90.47	57.54	91.96	65.10	58.98	59.80	58.59	65.50
716	372	8	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	88.46	62.56	63.61	43.72	88.97	90.42	87.78	60.78	84.99	89.48	60.39	57.76	89.44	57.22	59.08	85.94	59.34	60.60	57.05	92.21	87.43	58.21	58.15	60.01	59.02
896	372	8	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	88.57	61.95	61.41	63.75	86.41	85.71	89.53	60.79	43.72	83.37	59.06	59.69	88.95	58.75	62.96	90.56	59.49	61.76	57.81	56.14	59.05	62.53	58.91	59.20	61.40
899	372	8	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	85.92	61.84	89.24	88.21	61.23	57.30	88.03	61.34	60.66	60.41	56.87	57.80	91.44	59.93	61.84	89.73	57.28	91.76	58.30	93.22	58.34	57.87	60.68	59.20	58.16
908	372	8	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	90.32	89.64	64.51	60.15	88.21	91.68	84.95	59.55	61.58	89.38	87.44	88.52	89.54	57.91	60.00	91.01	57.56	61.01	83.96	54.85	59.92	58.83	59.53	58.03	56.89
911	372	8	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	87.15	61.02	89.21	89.40	89.51	59.47	90.29	61.72	89.72	61.05	91.42	92.42	90.91	91.60	59.14	92.11	61.52	92.24	57.28	59.07	62.80	60.77	58.26	49.09	59.96
1100	372	8	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	87.45	64.13	62.34	62.60	63.56	91.34	89.45	62.28	60.89	60.11	59.25	59.21	92.45	60.30	61.26	89.90	60.64	60.42	92.11	86.69	59.35	59.96	56.95	59.66	49.04
1103	372	8	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	63.03	89.71	63.72	61.20	62.54	91.95	87.41	60.60	61.47	89.72	55.37	60.25	88.48	60.93	60.43	90.67	59.64	59.23	93.43	58.25	56.40	58.01	58.74	59.47	59.23
140	372	8	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	89.47	61.30	61.12	61.95	87.38	90.42	88.83	61.92	60.35	81.76	88.96	57.45	91.40	60.55	60.78	88.80	92.48	60.05	58.25	58.30	56.84	62.01	89.98	59.06	57.26
323	372	8	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	88.35	61.23	86.23	60.61	59.33	89.55	90.24	59.74	60.41	60.76	59.04	56.87	88.98	58.23	58.85	59.00	59.01	90.92	93.15	56.00	58.99	57.60	59.09	60.54	60.10
335	372	8	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	88.53	59.96	60.22	61.27	89.95	58.07	90.72	60.70	90.41	90.37	58.59	60.34	82.30	57.75	59.59	91.46	60.03	55.97	57.65	55.78	59.52	61.48	55.21	58.71	59.36
527	372	8	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	87.89	62.31	62.06	62.19	62.80	59.24	88.41	61.47	91.55	85.14	60.77	59.33	90.96	60.50	60.55	88.10	58.87	60.20	58.57	58.00	88.60	58.49	59.77	59.62	56.70

Size of the All data:  (100, 28)
Size of the Sig data:  (24, 28)

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params1 = dt_valid.loc[dt_valid_sub1.index][['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
optimal_params = optimal_params1.reset_index()
optimal_params.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_1 = pd.merge(optimal_params, dt_test, on=['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg'])
test_data_1_sub = test_data_1[list(test_data_1.columns[test_data_1.columns.str.startswith('mAP_test')])]

Equivalent test data

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_zero	mAP_test_zero_2	mAP_test_zero_3	mAP_test_zero_4	mAP_test_zero_5	mAP_test_zero_6	mAP_test_zero_7	mAP_test_zero_8	mAP_test_zero_9	mAP_test_zero_10	mAP_test_zero_11	mAP_test_zero_12	mAP_test_zero_13	mAP_test_zero_14	mAP_test_zero_15	mAP_test_zero_16	mAP_test_zero_17	mAP_test_zero_18	mAP_test_zero_19	mAP_test_zero_20	mAP_test_zero_21	mAP_test_zero_22	mAP_test_zero_23	mAP_test_zero_25	mAP_test_zero_26
4	372	8	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	58.28	59.36	61.75	60.38	60.75	59.40	58.88	61.50	58.12	61.65	50.99	59.48	50.25	60.09	60.45	52.21	60.78	59.77	59.47	61.65	61.61	62.72	62.12	61.42	61.71
17	372	8	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	58.67	60.27	60.20	61.03	59.32	60.02	61.50	61.49	59.92	61.77	59.33	61.58	59.18	86.25	59.51	59.54	60.43	61.45	61.53	59.63	63.12	61.59	62.25	61.80	62.13
8	372	8	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	58.08	61.37	58.97	58.38	60.80	62.20	58.41	89.37	56.44	61.50	58.89	59.20	58.43	60.99	60.41	59.68	59.93	58.89	60.31	60.53	61.24	62.19	62.16	61.55	60.72
20	372	8	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	60.25	59.98	58.51	60.25	60.75	59.67	58.80	59.26	58.90	48.12	50.01	59.15	59.59	47.95	60.58	49.59	60.47	60.88	58.70	59.34	60.63	62.16	60.18	62.50	61.10
22	372	8	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	56.80	61.84	57.16	61.15	61.08	48.12	63.56	60.06	58.39	60.68	55.46	60.97	60.51	60.13	59.78	57.69	59.59	57.67	58.96	59.85	61.95	61.83	62.96	62.88	61.44
18	372	8	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	59.05	62.10	55.96	60.60	60.28	59.32	59.52	60.16	57.15	60.28	59.88	62.70	87.16	60.08	48.12	60.37	59.74	60.08	60.77	61.29	60.79	62.25	62.69	62.20	61.63
10	372	8	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	57.17	61.06	58.92	61.99	60.47	60.84	57.91	62.25	61.19	58.02	62.79	59.37	57.89	60.58	60.81	59.02	60.95	50.61	60.48	59.35	62.25	62.78	62.95	63.48	62.10
15	372	8	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	57.49	60.41	60.37	61.26	57.29	61.01	59.19	59.88	60.64	60.63	60.94	60.24	57.44	60.33	60.83	50.08	60.34	57.60	59.32	51.34	61.52	59.82	61.81	61.51	62.18
13	372	8	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	59.53	61.03	61.86	60.14	59.80	59.26	59.30	62.09	60.34	61.14	61.59	61.99	59.24	61.18	60.86	58.80	60.32	61.80	60.11	61.11	63.47	62.84	62.53	62.18	61.89
5	372	8	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	59.39	61.99	58.51	61.33	61.43	51.74	59.62	61.93	58.40	58.62	84.92	61.70	49.68	60.50	60.58	61.39	59.85	59.62	50.60	61.87	62.20	62.14	62.58	61.96	60.89
14	372	8	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	58.88	62.32	56.55	60.97	61.41	60.92	88.69	61.11	57.61	60.22	62.32	59.93	58.31	62.18	61.58	60.71	60.72	58.94	60.46	50.83	60.35	49.51	62.36	62.52	61.00
7	372	8	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	59.75	61.52	59.26	62.32	59.41	62.08	58.77	61.93	59.10	61.42	59.79	62.18	62.16	61.21	61.42	65.45	60.41	59.48	61.53	59.45	60.03	60.46	61.95	61.82	62.40
9	372	8	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	61.16	61.87	61.27	59.81	61.93	60.11	59.64	61.76	60.67	61.39	60.33	62.39	48.50	60.90	61.75	59.43	61.49	59.45	62.97	59.04	60.33	62.55	62.32	62.42	61.84
3	372	8	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	59.45	61.99	61.92	48.12	59.52	60.28	58.03	61.82	49.18	58.63	61.99	62.00	59.42	61.82	60.75	51.35	61.10	60.51	61.65	59.06	59.87	62.26	61.49	62.49	61.76
23	372	8	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	58.90	61.42	61.01	62.00	59.72	90.60	57.93	61.20	48.12	49.79	61.53	60.11	57.97	61.67	61.07	59.29	62.18	61.67	61.25	59.15	61.79	62.50	62.00	62.89	62.17
6	372	8	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	55.40	61.76	59.07	59.73	61.54	59.14	59.78	62.11	62.00	60.96	59.71	61.04	59.04	61.46	62.34	56.88	61.58	58.35	61.65	58.99	60.92	62.45	62.07	62.63	62.41
1	372	8	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	57.79	59.56	61.55	60.44	58.19	59.95	51.41	62.00	61.36	59.18	51.55	51.80	60.01	61.70	62.25	59.73	61.00	61.46	88.06	59.06	61.43	61.60	61.47	62.61	62.49
0	372	8	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	59.57	62.00	59.06	59.52	59.22	61.90	59.49	60.91	59.36	62.32	60.22	59.95	59.91	58.42	62.28	59.05	60.03	59.22	62.90	60.65	61.57	62.05	62.09	48.62	62.01
11	372	8	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	59.69	61.55	60.87	61.31	60.72	59.48	58.97	61.70	61.51	61.44	61.85	61.58	59.52	61.41	61.70	57.64	60.82	62.09	59.37	52.38	62.90	62.33	62.30	62.56	46.00
12	372	8	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	61.88	59.48	61.29	60.75	61.99	58.62	59.27	61.15	60.92	59.12	59.17	61.99	60.19	60.59	60.40	59.33	61.36	60.98	59.95	61.87	62.46	62.69	62.56	62.08	62.24
2	372	8	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	59.00	61.40	61.28	61.15	59.77	59.80	59.23	60.44	60.77	82.94	49.80	60.17	59.73	61.43	61.99	48.32	58.81	61.99	61.25	61.15	61.60	61.82	59.63	61.39	62.45
19	372	8	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	57.83	61.81	59.17	61.87	60.13	56.31	58.72	61.65	60.51	60.54	61.59	61.65	59.34	60.10	61.59	59.49	60.88	57.91	58.61	60.58	61.90	62.05	61.66	61.73	61.84
21	372	8	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	57.96	61.45	61.17	61.87	59.53	60.65	58.88	62.13	58.08	58.25	61.59	61.25	85.74	61.29	60.83	59.02	60.02	50.06	60.74	59.62	61.72	61.51	63.37	61.99	61.93
16	372	8	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	59.57	61.65	62.06	61.97	61.93	60.21	57.26	61.17	59.84	90.67	61.93	61.84	59.44	62.08	61.88	50.58	60.67	62.08	61.56	60.65	60.46	62.85	62.18	62.57	62.59

Size of the test data:  (24, 33)

Difference

diff = dt[['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
for i, cols in enumerate(zip(dt.columns,test_data_1.columns)):   
   diff[f'col_diff_{i}'] = (np.array(dt[cols[0]]) - np.array(test_data_1[cols[1]]))*-1

# Function to count negatives in a row
def count_negatives(row):
    return sum(1 for value in row if isinstance(value, (int, float)) and value < 0)

diff['negative_count'] = diff.apply(count_negatives, axis=1)
diff.sort_values(by =["negative_count"],inplace = True, ascending=False)

html_table = diff.to_html(index=True)

# Wrap in a scrollable div
scrollable_table = f"""
<div style="height: 400px; width: 100%; overflow-x: auto; overflow-y: auto;">
    {html_table}
</div>
"""
# Display the scrollable table
display(HTML(scrollable_table))

# 
# show(diff)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	col_diff_0	col_diff_1	col_diff_2	col_diff_3	col_diff_4	col_diff_5	col_diff_6	col_diff_7	col_diff_8	col_diff_9	col_diff_10	col_diff_11	col_diff_12	col_diff_13	col_diff_14	col_diff_15	col_diff_16	col_diff_17	col_diff_18	col_diff_19	col_diff_20	col_diff_21	col_diff_22	col_diff_23	col_diff_24	col_diff_25	col_diff_26	col_diff_27	col_diff_28	col_diff_29	col_diff_30	col_diff_31	col_diff_32	negative_count
1219	372	8	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.38	-29.08	-0.56	-1.29	0.25	-31.44	-28.32	-1.92	-31.24	-1.17	-35.76	-31.22	-5.78	-0.75	-0.21	-32.21	1.43	-1.56	0.44	1.82	-2.21	1.05	1.81	-2.20	2.31	18
1027	372	8	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-4.43	-1.30	-28.24	0.33	-4.42	-31.32	-28.90	-0.71	-2.31	4.40	-39.95	-32.21	-1.48	-37.44	1.98	-6.88	-0.27	1.99	-32.24	3.31	-0.54	1.94	-1.66	7.52	-0.13	18
79	372	8	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.37	-0.39	-30.29	-1.24	-1.07	-33.10	-28.74	-1.24	-32.40	-1.80	-32.21	3.03	2.48	-1.35	4.40	-0.22	-0.11	-0.92	1.51	2.99	-3.13	-0.02	3.25	0.15	2.95	17
911	372	8	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.58	0.98	-30.15	-29.88	-30.29	2.43	-30.80	-0.81	-30.36	1.27	-31.20	-32.47	-31.00	-33.18	3.14	-33.06	-1.49	-33.02	5.62	1.58	-1.23	1.28	3.83	-0.47	2.05	16
704	372	8	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-4.00	-0.66	-1.52	-29.08	-1.33	0.93	-6.97	0.76	-2.85	-2.87	-30.45	2.34	-40.29	-0.67	0.45	-32.28	0.83	-31.02	5.43	-32.92	-4.77	3.57	2.52	3.83	-3.66	16
256	372	8	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.03	-2.99	-28.71	-0.80	-3.45	0.73	-30.19	-0.20	-2.88	-29.32	2.40	1.54	-29.06	-0.69	-3.82	-33.22	1.63	-5.78	0.71	-33.11	3.07	2.17	2.05	6.32	2.19	15
451	372	8	0.1	32	0.00100	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.65	-2.43	-1.19	1.05	-31.09	2.20	-30.16	-28.94	-1.16	-0.60	1.34	1.99	-31.97	1.14	-0.16	-5.53	0.66	-32.38	0.81	-36.18	-1.64	-1.37	1.66	2.61	2.03	15
1231	372	8	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.16	-2.27	-1.93	-0.67	-1.57	-31.48	-0.83	1.33	-2.81	-0.65	-31.88	0.93	-33.36	3.23	-3.00	-31.76	0.09	-0.07	1.21	2.53	2.82	-0.88	0.20	0.66	1.82	15
1103	372	8	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-1.15	-30.23	-2.43	-0.45	-0.55	-33.33	-28.14	0.55	-0.55	-30.60	3.80	1.74	-28.29	-0.34	-0.03	-31.34	1.72	1.75	-33.48	3.62	6.06	4.68	3.82	2.61	3.01	14
67	372	8	0.1	32	0.00100	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.12	-0.68	-29.68	-0.37	0.60	4.40	7.28	-1.15	-28.77	-3.48	-32.79	2.29	-0.81	1.39	0.73	-32.20	-1.56	-30.32	-33.25	3.81	1.66	1.60	3.85	3.15	1.93	13
1100	372	8	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.76	-2.58	-1.47	-1.29	-2.84	-31.86	-30.48	-0.58	0.62	1.33	2.60	2.37	-32.93	1.11	0.44	-32.26	0.18	1.67	-32.74	-34.31	3.55	2.37	5.35	2.90	-3.04	13
716	372	8	0.1	64	0.00010	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.01	-0.57	-1.69	4.40	-29.45	-30.14	-29.75	1.04	-35.81	-30.85	1.60	4.24	-30.02	4.60	1.67	-34.59	1.76	-0.09	4.60	-33.15	-27.56	4.05	3.34	2.48	2.74	13
140	372	8	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.47	0.10	0.16	-0.80	-27.61	-30.62	-29.60	-1.48	0.42	1.18	-39.16	2.72	-31.67	0.88	1.21	-40.48	-33.67	1.94	3.00	2.85	4.76	-0.19	-30.35	2.33	5.19	12
908	372	8	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-32.53	-30.08	-2.96	0.29	-30.02	-31.73	-33.54	2.45	-0.22	-30.20	-35.89	-36.72	-29.53	3.79	2.25	-31.28	3.44	0.45	4.10	4.21	1.51	2.77	1.94	4.58	5.60	12
896	372	8	0.1	64	0.00010	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.67	-0.53	-0.40	-1.75	-26.69	4.89	-31.60	0.41	4.40	-33.58	2.47	0.42	-30.98	2.92	-1.89	-31.27	2.69	-0.09	3.44	3.01	2.74	-0.03	3.09	3.69	0.77	12
1487	372	8	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.72	-2.91	-30.54	0.13	-29.81	3.23	-29.92	2.87	-31.57	-1.40	-32.52	2.08	1.43	0.52	1.40	19.64	-0.70	-31.12	1.63	2.57	-29.40	2.35	4.28	3.58	3.80	11
835	372	8	0.1	32	0.00010	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.39	-1.04	-30.54	-30.80	-2.59	1.19	-31.15	6.11	-30.45	-0.05	-32.93	2.51	-31.75	0.79	1.70	-1.54	0.93	0.87	3.45	1.55	0.49	4.08	1.77	2.06	3.06	11
1472	372	8	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.59	1.31	-29.61	-0.52	-0.68	0.32	2.23	0.84	-32.61	-1.99	2.40	3.62	-34.20	2.36	1.69	1.79	1.17	-32.73	1.80	-39.64	-4.36	-6.23	2.73	1.36	1.63	11
1292	372	8	0.1	64	0.00001	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.81	0.46	-30.08	0.52	0.46	-33.82	-1.42	2.64	-32.07	-32.15	-1.51	3.88	-33.62	1.91	1.70	2.93	2.08	-30.66	-36.98	5.54	3.94	4.95	6.85	4.63	3.39	10
1280	372	8	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.71	0.22	-0.00	-30.23	-28.62	-32.30	-29.14	1.42	-1.24	-1.70	-0.12	2.98	-31.67	1.52	1.45	-32.41	2.21	2.58	2.31	2.79	2.92	6.81	2.01	2.75	1.83	10
527	372	8	0.1	64	0.00100	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.32	-0.66	-0.00	-0.22	-0.87	0.97	-31.15	-0.30	-31.71	5.53	1.16	2.51	-31.52	1.58	1.33	-37.52	1.80	1.88	2.99	2.65	-28.14	4.36	2.41	2.95	5.89	10
899	372	8	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.52	-0.08	-30.17	-28.48	0.31	1.84	-28.25	0.77	1.34	0.55	2.84	3.24	-32.40	1.53	0.50	-32.85	4.30	-33.41	3.35	-34.23	2.58	4.58	1.39	3.43	4.25	9
323	372	8	0.1	64	0.00100	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.52	0.58	-27.06	1.26	0.80	-33.24	-31.52	1.91	0.10	-0.22	2.55	4.78	-29.64	1.87	2.74	0.49	1.87	-33.01	-34.54	4.58	2.91	4.45	2.57	1.19	1.74	8
335	372	8	0.1	64	0.00100	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.57	1.49	0.95	0.60	-30.42	2.58	-31.84	1.43	-32.33	-32.12	3.00	0.91	3.44	3.54	1.24	-32.44	-0.01	-5.91	3.09	3.84	2.20	0.03	8.16	3.28	2.57	8

Model performance on valid data

Average & Std Deviation of the Significant rows:

	mean	std
mAP_valid_zero	85.083333	8.102378
mAP_valid_zero_13	84.302917	11.544230
mAP_valid_zero_7	83.985833	10.433061
mAP_valid_zero_16	78.963750	16.183350
mAP_valid_zero_6	75.429167	16.893546
mAP_valid_zero_11	74.515833	15.713278
mAP_valid_zero_3	73.993750	13.070421
mAP_valid_zero_9	73.540833	15.209465
mAP_valid_zero_5	71.997083	13.128487
mAP_valid_zero_18	71.313333	15.385844
mAP_valid_zero_10	70.823750	13.858653
mAP_valid_zero_19	67.747500	15.032302
mAP_valid_zero_20	67.030417	15.581366
mAP_valid_zero_4	66.567917	12.437834
mAP_valid_zero_2	65.522083	9.230255
mAP_valid_zero_21	64.078333	9.708070
mAP_valid_zero_12	64.031667	12.279621
mAP_valid_zero_14	63.086667	9.277282
mAP_valid_zero_8	62.994583	7.218968
mAP_valid_zero_17	60.937083	6.812525
mAP_valid_zero_23	60.533750	6.490406
mAP_valid_zero_15	59.702083	3.717174
mAP_valid_zero_22	59.524167	2.030457
mAP_valid_zero_26	58.958333	2.780272
mAP_valid_zero_25	58.933750	2.699312

Model performance on Test data

We then apply the hyperparameters to the test set and average the results.

test_data_1['average_map'] = test_data_1_sub.apply(np.mean, axis=1)

test_data_1.sort_values(by=['average_map'], ascending= False,inplace=True)

	mean	std
mAP_test_zero_8	62.461250	5.788103
mAP_test_zero_23	62.070000	0.809852
mAP_test_zero_25	61.658333	2.826128
mAP_test_zero_10	61.636667	8.554690
mAP_test_zero_22	61.539583	2.660315
mAP_test_zero_21	61.504583	0.960570
mAP_test_zero_14	61.430833	5.968238
mAP_test_zero_19	61.341667	6.160799
mAP_test_zero_2	61.216250	0.888505
mAP_test_zero_26	61.205000	3.279959
mAP_test_zero_12	60.594167	2.168486
mAP_test_zero_15	60.573333	2.761872
mAP_test_zero_17	60.561250	0.733244
average_map	60.495000	0.834057
mAP_test_zero_6	60.484583	7.144606
mAP_test_zero_13	60.360417	8.747811
mAP_test_zero_4	60.347500	2.770020
mAP_test_zero_5	60.290833	1.183440
mAP_test_zero_7	60.115000	6.411016
mAP_test_zero_11	59.923750	6.710322
mAP_test_zero_3	59.905833	1.754772
mAP_test_zero_18	59.273333	3.101440
mAP_test_zero_20	59.101667	3.086252
mAP_test_zero	58.814167	1.396607
mAP_test_zero_9	58.688333	3.429270
mAP_test_zero_16	57.276667	4.416110

Model performance on validation data (Mann-Whitney U Test)

Statistical Significance:

Using the global mean maximum row as the reference, we perform the right-tailed t-test to identify significant hyperparameters.

pvalues_mw = []
pvalues_real_mw = []

for _,index in enumerate(dt_valid_sub.index):

  _, p_value_mw = stats.mannwhitneyu(max_row[:-1], dt_valid_sub.loc[index][:-1], alternative='greater')#, method = 'asymptotic')
  pvalues_real_mw.append(p_value_mw)
  pvalues_mw.append((p_value_mw >= 0.05)*1)
  

dt_valid_sub_mw['pvalues_mw'] = pvalues_real_mw
dt_valid_sub_mw['sig_mw'] = pvalues_mw

dt_valid_sub_mw1 = dt_valid_sub_mw.query('sig_mw == 1')


max_indexx = dt_valid_sub_mw1.iloc[:, :-3].mean().idxmax()

#print(sum(pvalues),max_indexx)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_valid_zero	mAP_valid_zero_2	mAP_valid_zero_3	mAP_valid_zero_4	mAP_valid_zero_5	mAP_valid_zero_6	mAP_valid_zero_7	mAP_valid_zero_8	mAP_valid_zero_9	mAP_valid_zero_10	mAP_valid_zero_11	mAP_valid_zero_12	mAP_valid_zero_13	mAP_valid_zero_14	mAP_valid_zero_15	mAP_valid_zero_16	mAP_valid_zero_17	mAP_valid_zero_18	mAP_valid_zero_19	mAP_valid_zero_20	mAP_valid_zero_21	mAP_valid_zero_22	mAP_valid_zero_23	mAP_valid_zero_25	mAP_valid_zero_26
1152	372	8	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	68.04	65.12	62.78	62.22	66.18	59.08	54.79	65.32	67.11	66.64	61.77	59.17	65.81	69.12	63.05	58.04	64.07	65.58	48.36	60.58	65.49	65.53	67.19	65.33	65.35
1164	372	8	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	68.03	61.94	66.68	63.39	64.62	62.24	56.91	64.55	66.55	64.90	61.15	60.15	65.30	62.63	65.09	64.77	65.28	80.75	60.78	58.40	62.09	64.49	65.55	68.00	68.40
1167	372	8	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	64.64	62.97	56.50	61.78	61.13	62.02	65.05	61.36	63.12	64.09	60.48	61.79	87.10	60.02	65.79	63.95	61.57	62.27	59.01	59.03	64.07	62.37	63.88	63.00	62.98
1344	372	8	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	71.39	70.84	68.74	72.69	65.30	66.23	52.13	70.63	69.59	69.62	63.59	62.70	67.55	66.27	66.65	53.07	69.81	67.78	43.72	65.17	70.13	69.81	66.47	66.03	70.97
1356	372	8	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	65.97	66.98	63.13	64.24	63.31	61.58	86.68	55.07	66.03	64.81	61.48	60.32	63.91	63.17	64.95	63.63	63.87	64.10	60.55	60.26	69.01	67.68	63.21	66.64	63.31
1359	372	8	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	65.93	63.97	62.80	64.96	65.91	56.07	55.57	65.26	87.18	64.97	60.02	60.39	56.84	64.63	63.03	63.34	63.20	64.10	61.48	59.69	66.77	68.82	62.23	64.56	64.68
576	372	8	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	67.51	66.81	66.12	66.51	74.73	61.66	57.16	67.62	69.41	68.02	62.68	66.25	70.24	65.14	65.76	63.59	68.62	65.56	61.68	62.39	67.30	66.41	69.03	66.25	67.93
579	372	8	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	58.28	62.01	65.54	65.76	61.46	60.81	64.73	61.38	59.04	64.41	59.11	59.78	62.34	59.18	64.06	64.62	64.73	61.73	60.59	59.86	66.04	64.22	64.29	67.28	62.98
588	372	8	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	85.53	62.01	64.33	63.83	85.90	60.27	65.88	62.63	63.15	63.80	60.50	61.18	62.77	61.20	65.79	60.62	60.07	63.47	60.67	58.51	63.57	64.35	61.92	64.46	62.49
771	372	8	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	64.96	61.40	61.01	64.98	64.66	59.58	63.07	62.66	64.15	61.40	60.84	59.65	87.68	62.64	62.34	63.41	62.27	61.58	57.99	59.19	64.23	63.72	62.18	66.20	66.17
780	372	8	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	65.52	65.53	64.55	61.45	64.51	59.20	54.21	64.36	64.08	66.78	62.93	58.25	61.61	61.76	61.80	62.59	61.05	63.72	59.31	59.69	62.18	63.18	63.13	62.75	62.09
783	372	8	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	66.41	64.26	63.32	64.49	64.73	61.80	66.34	64.61	64.06	62.51	59.15	59.03	64.50	62.32	63.75	67.05	65.42	63.31	59.42	57.43	67.05	63.80	67.74	65.61	67.35
960	372	8	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	65.91	64.19	56.86	63.70	60.83	64.22	64.98	62.42	63.36	54.32	55.39	63.42	64.12	63.08	60.96	63.25	60.98	64.39	58.98	63.46	67.42	64.95	63.46	66.85	65.81
972	372	8	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	65.10	67.66	68.03	66.09	64.82	62.07	65.64	65.81	65.91	64.17	60.81	60.65	54.60	64.27	65.55	64.51	65.68	67.47	60.60	61.30	64.23	63.97	60.88	64.32	62.25
15	372	8	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	65.42	65.02	64.94	65.93	64.71	61.95	78.06	57.81	63.08	65.07	62.43	62.91	65.24	64.06	64.44	61.44	62.85	68.90	59.89	59.04	64.92	67.46	65.79	66.72	68.90
207	372	8	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	67.68	68.79	68.78	67.94	67.85	66.43	69.18	64.75	66.58	64.38	67.38	63.50	67.26	64.36	66.40	71.82	69.49	66.18	65.72	63.18	64.90	71.23	67.92	68.07	68.20
387	372	8	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	68.43	66.34	57.79	62.97	56.31	43.72	79.09	78.38	63.47	65.84	61.40	59.78	89.08	60.16	63.19	53.95	63.36	64.43	57.12	61.72	63.84	61.40	64.75	67.49	60.03
396	372	8	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	67.38	63.05	66.69	66.94	65.87	59.19	56.12	63.72	66.38	64.80	57.94	65.64	59.34	86.56	61.13	65.46	64.13	66.36	60.93	66.19	65.04	64.76	62.84	67.44	64.85
1216	372	8	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	63.27	61.16	63.40	63.89	63.04	59.41	87.71	61.60	60.34	60.50	88.77	57.82	90.37	60.21	61.32	61.61	62.34	62.51	83.91	88.91	64.41	61.35	62.58	66.01	62.19
1219	372	8	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	88.66	88.44	62.31	61.67	60.50	90.84	87.20	63.42	89.36	62.82	86.75	90.70	56.03	60.84	60.66	84.42	59.35	61.33	59.03	59.83	63.82	61.67	60.31	63.62	59.40
1231	372	8	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	88.83	62.54	62.13	61.70	60.89	91.50	62.33	60.16	62.73	62.42	91.21	60.65	92.54	83.02	62.51	91.30	60.34	61.52	60.32	57.10	60.30	62.47	62.05	61.14	60.31
1420	372	8	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	57.91	63.17	64.46	64.01	63.21	90.04	66.62	65.11	62.63	61.18	59.39	60.85	89.39	90.74	62.02	60.22	62.09	62.39	59.10	58.94	61.44	61.37	61.12	68.22	59.79
1423	372	8	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	63.53	62.26	62.03	88.78	60.23	61.44	64.88	60.90	62.50	63.05	89.47	60.47	91.03	61.18	61.54	61.97	61.07	60.87	90.18	58.54	62.23	61.35	62.99	60.80	57.90
640	372	8	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	86.78	62.69	64.46	62.29	88.23	59.84	60.84	59.91	63.52	90.09	91.73	57.12	91.82	60.06	59.48	60.73	60.73	63.46	57.44	58.95	63.10	62.49	59.24	63.68	62.98
643	372	8	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	63.25	64.65	64.14	64.25	64.17	89.06	88.04	61.65	43.72	63.50	58.71	60.17	63.52	62.82	62.86	88.07	60.38	84.08	60.33	58.54	62.77	64.09	60.70	59.95	62.66
652	372	8	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	86.60	64.23	61.25	61.42	60.72	90.32	86.21	59.42	88.00	61.24	90.18	59.28	90.28	59.43	62.93	60.08	61.45	58.34	59.40	57.95	59.84	62.49	58.55	60.35	61.88
832	372	8	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	63.11	64.16	88.09	62.34	61.70	62.14	82.78	62.95	49.83	63.49	87.38	61.40	63.37	61.57	62.46	63.79	61.97	61.47	60.05	90.75	64.82	61.42	61.32	59.75	61.13
847	372	8	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	88.46	86.61	61.60	60.40	63.10	59.16	85.88	60.11	55.63	62.34	90.63	58.39	90.63	61.86	60.48	62.45	60.25	63.86	91.04	58.40	61.51	59.93	60.77	61.60	59.38
1024	372	8	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	64.28	61.86	71.58	61.55	63.15	59.53	88.43	60.36	60.28	62.11	59.09	56.64	78.46	89.16	63.99	77.15	59.31	89.65	60.28	56.91	66.13	64.23	59.63	58.79	61.52
1027	372	8	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	64.68	61.28	86.75	59.92	65.17	90.99	87.70	59.97	61.21	43.72	89.96	91.36	61.07	85.39	58.60	56.47	60.74	58.89	90.94	56.03	61.17	60.22	61.84	54.98	61.23
79	372	8	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	88.42	62.49	86.25	61.84	61.35	92.42	88.26	61.40	89.55	62.08	92.09	59.67	84.68	61.43	43.72	60.59	59.85	61.00	59.26	58.30	63.92	62.27	59.44	62.05	58.68
256	372	8	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	86.20	64.05	87.63	62.79	63.92	60.11	88.10	62.45	64.07	87.34	60.39	57.83	86.95	61.27	64.63	92.24	59.32	56.39	59.77	92.46	59.18	60.61	60.90	57.16	59.91
259	372	8	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	64.49	63.22	61.79	61.09	62.59	89.97	87.92	63.02	89.38	62.12	91.86	58.58	61.06	55.48	62.05	46.71	62.76	59.58	59.86	89.58	63.15	62.28	60.32	50.80	59.49
463	372	8	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	89.13	62.47	61.20	62.03	62.08	58.88	87.86	61.97	89.74	62.83	60.90	55.49	43.72	59.43	61.91	91.94	63.77	60.01	57.97	56.71	61.91	57.70	88.09	61.04	60.43
1280	372	8	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	89.24	60.81	61.86	90.37	88.42	91.56	88.44	60.67	61.58	62.84	61.71	59.01	90.91	59.66	59.41	91.21	58.11	59.22	57.80	58.32	60.55	56.03	60.52	59.43	60.06
1472	372	8	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	88.47	61.01	86.16	61.49	62.09	60.60	86.46	60.27	90.22	62.21	59.92	56.31	92.51	59.82	59.89	58.92	59.55	91.67	58.66	90.47	64.71	55.74	59.63	61.16	59.37
1487	372	8	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	87.47	64.43	89.80	62.19	89.22	58.85	88.69	59.06	90.67	62.82	92.31	60.10	60.73	60.69	60.02	45.81	61.11	90.60	59.90	56.88	89.43	58.11	57.67	58.24	58.60
704	372	8	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	65.16	62.53	62.79	88.89	63.26	59.18	66.61	61.00	63.52	64.26	90.78	60.05	88.79	61.57	61.30	91.71	60.66	90.47	57.54	91.96	65.10	58.98	59.80	58.59	65.50
899	372	8	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	85.92	61.84	89.24	88.21	61.23	57.30	88.03	61.34	60.66	60.41	56.87	57.80	91.44	59.93	61.84	89.73	57.28	91.76	58.30	93.22	58.34	57.87	60.68	59.20	58.16
908	372	8	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	90.32	89.64	64.51	60.15	88.21	91.68	84.95	59.55	61.58	89.38	87.44	88.52	89.54	57.91	60.00	91.01	57.56	61.01	83.96	54.85	59.92	58.83	59.53	58.03	56.89
911	372	8	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	87.15	61.02	89.21	89.40	89.51	59.47	90.29	61.72	89.72	61.05	91.42	92.42	90.91	91.60	59.14	92.11	61.52	92.24	57.28	59.07	62.80	60.77	58.26	49.09	59.96
1100	372	8	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	87.45	64.13	62.34	62.60	63.56	91.34	89.45	62.28	60.89	60.11	59.25	59.21	92.45	60.30	61.26	89.90	60.64	60.42	92.11	86.69	59.35	59.96	56.95	59.66	49.04
1103	372	8	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	63.03	89.71	63.72	61.20	62.54	91.95	87.41	60.60	61.47	89.72	55.37	60.25	88.48	60.93	60.43	90.67	59.64	59.23	93.43	58.25	56.40	58.01	58.74	59.47	59.23
140	372	8	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	89.47	61.30	61.12	61.95	87.38	90.42	88.83	61.92	60.35	81.76	88.96	57.45	91.40	60.55	60.78	88.80	92.48	60.05	58.25	58.30	56.84	62.01	89.98	59.06	57.26
320	372	8	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	87.94	64.65	62.17	61.04	62.46	57.04	86.61	61.29	90.47	61.19	59.23	43.72	91.18	59.52	61.15	90.43	59.74	91.46	60.84	58.32	59.05	63.77	60.16	60.63	57.86

Size of the All data:  (100, 28)
Size of the Sig data:  (45, 33)

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params_mw1 = dt_valid.loc[dt_valid_sub_mw1.index][['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
optimal_params_mw = optimal_params_mw1.reset_index()
optimal_params_mw.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_mw1 = pd.merge(optimal_params_mw, dt_test, on=['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg'])
test_data_1_sub_mw = test_data_mw1[list(test_data_mw1.columns[test_data_mw1.columns.str.startswith('mAP_test')])]

Equivalent test data

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	mAP_test_zero	mAP_test_zero_2	mAP_test_zero_3	mAP_test_zero_4	mAP_test_zero_5	mAP_test_zero_6	mAP_test_zero_7	mAP_test_zero_8	mAP_test_zero_9	mAP_test_zero_10	mAP_test_zero_11	mAP_test_zero_12	mAP_test_zero_13	mAP_test_zero_14	mAP_test_zero_15	mAP_test_zero_16	mAP_test_zero_17	mAP_test_zero_18	mAP_test_zero_19	mAP_test_zero_20	mAP_test_zero_21	mAP_test_zero_22	mAP_test_zero_23	mAP_test_zero_25	mAP_test_zero_26
40	372	8	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	59.73	59.62	62.79	61.30	60.03	58.65	52.91	60.43	61.35	61.33	59.69	59.64	61.78	58.81	61.14	64.98	60.50	59.73	45.06	58.91	61.75	61.18	61.35	61.42	61.80
33	372	8	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	60.26	59.06	60.26	59.35	59.11	59.30	49.40	59.68	58.74	58.74	60.43	59.14	61.41	61.28	59.82	59.72	60.46	59.21	60.37	57.95	62.32	59.23	62.12	60.46	60.80
41	372	8	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	59.63	60.48	50.31	61.49	60.60	59.64	61.77	59.36	61.15	60.84	60.79	59.54	58.83	59.18	61.36	59.91	60.02	59.35	59.65	60.15	60.02	61.32	61.47	61.44	61.43
25	372	8	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	58.95	58.94	61.11	58.71	61.64	57.22	49.99	59.41	59.66	60.07	58.48	57.83	60.52	60.60	61.13	51.60	59.03	61.36	48.12	57.87	59.69	60.80	61.63	61.91	62.05
32	372	8	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	59.76	60.13	60.04	60.33	60.05	59.82	57.98	50.93	60.43	60.33	62.12	59.57	59.23	59.48	60.05	60.79	59.29	58.64	60.51	59.19	60.72	60.68	61.87	60.68	61.53
35	372	8	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	59.03	61.01	60.34	60.32	61.74	49.96	49.14	59.19	56.83	59.25	62.22	60.12	49.87	59.66	59.64	60.04	59.62	59.50	59.19	59.86	60.89	59.35	61.98	61.48	60.95
21	372	8	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	60.44	61.24	59.49	60.16	61.74	60.65	47.45	59.84	59.88	59.75	59.42	59.23	59.25	65.29	60.63	57.96	59.14	58.87	59.39	59.35	61.48	60.57	60.60	61.10	60.83
43	372	8	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	48.60	60.56	59.58	61.16	59.34	60.09	59.36	60.90	49.45	61.40	60.48	59.54	61.29	59.47	59.97	58.57	60.46	59.23	60.17	59.31	60.95	61.78	60.88	60.74	61.93
34	372	8	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	56.41	62.26	59.97	59.18	58.01	59.13	60.52	59.74	58.56	59.96	59.80	61.50	59.41	59.37	60.85	60.52	59.27	59.80	60.03	58.47	60.93	62.67	62.80	62.53	62.48
39	372	8	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	60.65	60.53	59.46	60.30	59.33	59.11	58.86	59.18	59.91	58.47	60.42	58.42	57.63	59.61	60.41	59.84	59.15	58.18	58.54	58.78	61.00	62.26	62.76	61.68	61.80
44	372	8	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	59.99	60.00	60.11	59.55	58.38	58.81	51.02	61.32	59.51	60.53	60.17	59.44	60.04	59.38	60.91	59.38	59.46	59.86	59.16	59.99	62.31	60.74	61.60	61.28	62.15
37	372	8	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	60.49	59.77	59.33	60.97	60.61	59.39	59.78	60.49	58.70	60.10	59.75	59.84	59.18	60.14	59.75	60.20	58.83	59.11	58.84	58.61	60.36	61.40	60.32	61.73	60.09
42	372	8	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	60.69	58.84	48.99	60.55	60.04	59.87	61.04	58.84	60.07	50.32	51.19	60.31	59.53	60.13	59.48	59.73	62.96	58.44	59.54	59.23	61.64	62.09	62.32	67.63	61.05
36	372	8	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	59.24	59.36	60.87	61.00	60.42	59.96	61.00	64.32	60.18	59.34	59.26	58.99	52.63	59.28	61.29	58.31	59.16	59.37	60.65	60.05	61.28	61.99	62.95	60.19	61.50
31	372	8	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	60.25	58.62	60.35	60.99	59.26	60.33	49.61	48.88	60.28	60.67	60.22	58.65	61.05	58.88	59.94	59.17	60.63	59.96	59.96	59.63	61.82	61.43	61.69	61.15	60.55
18	372	8	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	60.28	60.98	59.21	60.77	60.90	60.15	59.43	61.04	60.33	60.30	60.48	60.70	60.91	60.93	60.23	57.42	58.95	60.43	60.18	59.77	61.30	59.53	60.69	61.06	60.09
38	372	8	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	60.28	60.99	49.50	59.37	50.61	48.12	49.46	35.10	60.22	60.64	59.80	59.42	57.88	58.88	60.57	62.76	59.64	59.94	58.54	59.67	61.56	60.81	60.91	60.39	62.57
30	372	8	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	59.09	60.25	61.54	60.60	59.90	48.56	50.43	59.49	59.00	59.19	58.53	59.02	46.16	59.23	59.25	58.64	60.31	60.21	58.66	58.28	61.54	62.59	61.58	60.78	61.88
17	372	8	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	60.29	58.88	58.32	59.47	60.96	58.90	58.26	59.61	58.79	60.37	51.35	59.47	58.42	58.62	59.09	61.54	60.81	62.28	87.22	56.55	61.69	61.39	61.60	61.30	61.64
3	372	8	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	58.28	59.36	61.75	60.38	60.75	59.40	58.88	61.50	58.12	61.65	50.99	59.48	50.25	60.09	60.45	52.21	60.78	59.77	59.47	61.65	61.61	62.72	62.12	61.42	61.71
12	372	8	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	58.67	60.27	60.20	61.03	59.32	60.02	61.50	61.49	59.92	61.77	59.33	61.58	59.18	86.25	59.51	59.54	60.43	61.45	61.53	59.63	63.12	61.59	62.25	61.80	62.13
29	372	8	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	49.86	61.27	59.20	61.63	60.02	58.28	59.78	61.16	60.09	59.62	62.13	61.93	58.32	59.24	60.30	59.66	60.67	60.68	61.03	60.44	61.77	62.32	62.36	60.26	62.34
23	372	8	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	61.92	61.03	61.37	57.94	60.93	61.45	60.20	60.77	61.42	61.28	55.56	61.59	58.63	61.31	60.53	60.24	60.29	60.38	57.35	61.53	60.75	62.25	62.13	61.45	60.95
20	372	8	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	57.99	60.21	59.58	61.01	57.73	60.32	58.03	59.93	61.73	58.42	59.64	58.67	58.51	60.82	60.33	59.90	60.56	60.29	59.71	60.40	61.68	62.04	62.68	61.52	61.60
28	372	8	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	61.36	60.82	61.13	61.09	61.63	59.74	58.58	61.39	48.12	60.83	60.42	60.58	60.42	60.41	59.95	57.77	60.18	82.77	60.06	60.17	60.98	61.57	61.66	62.07	61.23
16	372	8	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	56.36	61.68	60.05	61.44	60.10	57.62	56.24	61.40	58.61	61.16	59.15	61.47	58.97	60.51	61.25	60.77	60.48	60.37	61.59	59.21	62.44	61.45	61.79	61.78	61.82
26	372	8	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	60.48	61.67	58.67	59.36	60.96	60.98	58.86	60.32	53.08	61.37	47.75	61.53	60.41	60.57	60.37	59.78	60.35	60.99	62.40	57.79	61.06	62.22	62.17	61.79	62.62
15	372	8	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	58.93	57.38	60.72	58.88	60.38	60.76	56.87	61.05	48.86	61.25	59.16	60.53	58.98	60.47	60.42	59.65	59.80	61.07	58.03	60.54	62.27	60.86	60.80	61.32	60.75
22	372	8	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	61.08	61.25	62.54	59.50	61.44	62.71	57.80	58.81	60.15	60.77	60.37	50.57	47.44	58.54	57.86	75.80	58.64	56.80	60.82	58.25	61.70	60.98	62.49	62.74	61.97
14	372	8	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	60.25	59.98	58.51	60.25	60.75	59.67	58.80	59.26	58.90	48.12	50.01	59.15	59.59	47.95	60.58	49.59	60.47	60.88	58.70	59.34	60.63	62.16	60.18	62.50	61.10
13	372	8	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	59.05	62.10	55.96	60.60	60.28	59.32	59.52	60.16	57.15	60.28	59.88	62.70	87.16	60.08	48.12	60.37	59.74	60.08	60.77	61.29	60.79	62.25	62.69	62.20	61.63
7	372	8	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	57.17	61.06	58.92	61.99	60.47	60.84	57.91	62.25	61.19	58.02	62.79	59.37	57.89	60.58	60.81	59.02	60.95	50.61	60.48	59.35	62.25	62.78	62.95	63.48	62.10
24	372	8	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	61.16	60.37	61.37	60.34	61.22	59.25	57.47	61.44	57.71	59.51	57.43	61.43	61.23	49.19	60.82	75.41	59.66	61.18	60.09	59.57	61.78	62.14	61.94	45.25	61.80
27	372	8	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	57.99	60.64	60.62	62.36	60.89	58.51	57.33	60.08	58.88	60.71	61.62	58.61	48.12	59.93	60.71	59.52	61.02	60.08	61.09	59.38	62.26	62.76	60.03	62.28	61.94
10	372	8	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	59.53	61.03	61.86	60.14	59.80	59.26	59.30	62.09	60.34	61.14	61.59	61.99	59.24	61.18	60.86	58.80	60.32	61.80	60.11	61.11	63.47	62.84	62.53	62.18	61.89
11	372	8	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	58.88	62.32	56.55	60.97	61.41	60.92	88.69	61.11	57.61	60.22	62.32	59.93	58.31	62.18	61.58	60.71	60.72	58.94	60.46	50.83	60.35	49.51	62.36	62.52	61.00
5	372	8	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	59.75	61.52	59.26	62.32	59.41	62.08	58.77	61.93	59.10	61.42	59.79	62.18	62.16	61.21	61.42	65.45	60.41	59.48	61.53	59.45	60.03	60.46	61.95	61.82	62.40
6	372	8	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	61.16	61.87	61.27	59.81	61.93	60.11	59.64	61.76	60.67	61.39	60.33	62.39	48.50	60.90	61.75	59.43	61.49	59.45	62.97	59.04	60.33	62.55	62.32	62.42	61.84
4	372	8	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	55.40	61.76	59.07	59.73	61.54	59.14	59.78	62.11	62.00	60.96	59.71	61.04	59.04	61.46	62.34	56.88	61.58	58.35	61.65	58.99	60.92	62.45	62.07	62.63	62.41
1	372	8	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	57.79	59.56	61.55	60.44	58.19	59.95	51.41	62.00	61.36	59.18	51.55	51.80	60.01	61.70	62.25	59.73	61.00	61.46	88.06	59.06	61.43	61.60	61.47	62.61	62.49
0	372	8	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	59.57	62.00	59.06	59.52	59.22	61.90	59.49	60.91	59.36	62.32	60.22	59.95	59.91	58.42	62.28	59.05	60.03	59.22	62.90	60.65	61.57	62.05	62.09	48.62	62.01
8	372	8	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	59.69	61.55	60.87	61.31	60.72	59.48	58.97	61.70	61.51	61.44	61.85	61.58	59.52	61.41	61.70	57.64	60.82	62.09	59.37	52.38	62.90	62.33	62.30	62.56	46.00
9	372	8	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	61.88	59.48	61.29	60.75	61.99	58.62	59.27	61.15	60.92	59.12	59.17	61.99	60.19	60.59	60.40	59.33	61.36	60.98	59.95	61.87	62.46	62.69	62.56	62.08	62.24
2	372	8	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	59.00	61.40	61.28	61.15	59.77	59.80	59.23	60.44	60.77	82.94	49.80	60.17	59.73	61.43	61.99	48.32	58.81	61.99	61.25	61.15	61.60	61.82	59.63	61.39	62.45
19	372	8	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	57.74	61.29	61.62	61.93	60.77	60.06	56.64	61.70	58.40	61.11	61.19	48.12	58.59	60.72	60.79	58.82	60.57	58.51	61.65	61.36	61.91	61.68	62.26	62.51	62.12

Size of the test data:  (45, 33)

Difference

diff_mw = dt_mw[['input_size', 'output_size', 'learning_rate', 'batch_size', 'alpha', 'margin', 'l1_reg', 'l2_reg']]
for i, cols in enumerate(zip(dt_mw.columns,test_data_mw1.columns)):   
   diff_mw[f'col_diff_{i}'] = (np.array(dt_mw[cols[0]]) - np.array(test_data_mw1[cols[1]]))*-1

# Function to count negatives in a row
def count_negatives(row):
    return sum(1 for value in row if isinstance(value, (int, float)) and value < 0)

diff_mw['negative_count'] = diff_mw.apply(count_negatives, axis=1)

diff_mw.sort_values(by =["negative_count"],inplace = True, ascending=False)


html_table = diff_mw.to_html(index=True)

# Wrap in a scrollable div
scrollable_table = f"""
<div style="height: 400px; width: 100%; overflow-x: auto; overflow-y: auto;">
    {html_table}
</div>
"""
# Display the scrollable table
display(HTML(scrollable_table))

# show(diff)

	input_size	output_size	learning_rate	batch_size	alpha	margin	l1_reg	l2_reg	col_diff_0	col_diff_1	col_diff_2	col_diff_3	col_diff_4	col_diff_5	col_diff_6	col_diff_7	col_diff_8	col_diff_9	col_diff_10	col_diff_11	col_diff_12	col_diff_13	col_diff_14	col_diff_15	col_diff_16	col_diff_17	col_diff_18	col_diff_19	col_diff_20	col_diff_21	col_diff_22	col_diff_23	col_diff_24	col_diff_25	col_diff_26	col_diff_27	col_diff_28	col_diff_29	col_diff_30	col_diff_31	col_diff_32	negative_count
207	372	8	0.1	16	0.00100	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-7.40	-7.81	-9.57	-7.17	-6.95	-6.28	-9.75	-3.71	-6.25	-4.08	-6.90	-2.80	-6.35	-3.43	-6.17	-14.40	-10.54	-5.75	-5.54	-3.41	-3.60	-11.70	-7.23	-7.01	-8.11	25
1164	372	8	0.1	16	0.00001	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-7.77	-2.88	-6.42	-4.04	-5.51	-2.94	-7.51	-4.87	-7.81	-6.16	-0.72	-1.01	-3.89	-1.35	-5.27	-5.05	-4.82	-21.54	-0.41	-0.45	0.23	-5.26	-3.43	-7.54	-7.60	24
1344	372	8	0.1	16	0.00001	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-12.44	-11.90	-7.63	-13.98	-3.66	-9.01	-2.14	-11.22	-9.93	-9.55	-5.11	-4.87	-7.03	-5.67	-5.52	-1.47	-10.78	-6.42	4.40	-7.30	-10.44	-9.01	-4.84	-4.12	-8.92	24
1356	372	8	0.1	16	0.00001	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-6.21	-6.85	-3.09	-3.91	-3.26	-1.76	-28.70	-4.14	-5.60	-4.48	0.64	-0.75	-4.68	-3.69	-4.90	-2.84	-4.58	-5.46	-0.04	-1.07	-8.29	-7.00	-1.34	-5.96	-1.78	24
576	372	8	0.1	16	0.00010	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-7.07	-5.57	-6.63	-6.35	-12.99	-1.01	-9.71	-7.78	-9.53	-8.27	-3.26	-7.02	-10.99	0.15	-5.13	-5.63	-9.48	-6.69	-2.29	-3.04	-5.82	-5.84	-8.43	-5.15	-7.10	24
396	372	8	0.1	16	0.00100	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-8.29	-2.80	-5.15	-6.34	-5.97	-10.63	-5.69	-4.23	-7.38	-5.61	0.59	-6.62	-13.18	-27.33	-1.88	-6.82	-3.82	-6.15	-2.27	-7.91	-3.50	-2.17	-1.26	-6.66	-2.97	24
972	372	8	0.1	16	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-5.86	-8.30	-7.16	-5.09	-4.40	-2.11	-4.64	-1.49	-5.73	-4.83	-1.55	-1.66	-1.97	-4.99	-4.26	-6.20	-6.52	-8.10	0.05	-1.25	-2.95	-1.98	2.07	-4.13	-0.75	23
15	372	8	0.1	16	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-5.17	-6.40	-4.59	-4.94	-5.45	-1.62	-28.45	-8.93	-2.80	-4.40	-2.21	-4.26	-4.19	-5.18	-4.50	-2.27	-2.22	-8.94	0.07	0.59	-3.10	-6.03	-4.10	-5.57	-8.35	23
1359	372	8	0.1	16	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-6.90	-2.96	-2.46	-4.64	-4.17	-6.11	-6.43	-6.07	-30.35	-5.72	2.20	-0.27	-6.97	-4.97	-3.39	-3.30	-3.58	-4.60	-2.29	0.17	-5.88	-9.47	-0.25	-3.08	-3.73	23
579	372	8	0.1	16	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-9.68	-1.45	-5.96	-4.60	-2.12	-0.72	-5.37	-0.48	-9.59	-3.01	1.37	-0.24	-1.05	0.29	-4.09	-6.05	-4.27	-2.50	-0.42	-0.55	-5.09	-2.44	-3.41	-6.54	-1.05	23
771	372	8	0.1	16	0.00010	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-4.31	-0.87	-1.55	-4.68	-5.33	-0.47	-4.21	-3.48	-4.24	-2.93	-0.42	-1.23	-30.05	-3.03	-1.93	-3.57	-3.12	-3.40	0.55	-0.41	-3.23	-1.46	0.58	-4.52	-4.37	23
1216	372	8	0.1	32	0.00001	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-2.98	-2.28	-5.08	-4.42	-2.08	-0.51	-29.45	-1.99	-1.55	-0.13	-37.42	1.65	-31.95	-1.59	-2.23	-0.07	-1.53	-0.23	3.31	-32.36	-2.72	0.04	-0.98	-4.71	-0.55	22
1152	372	8	0.1	16	0.00001	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-8.31	-5.50	0.01	-0.92	-6.15	-0.43	-1.88	-4.89	-5.76	-5.31	-2.08	0.47	-4.03	-10.31	-1.91	6.94	-3.57	-5.85	-3.30	-1.67	-3.74	-4.35	-5.84	-3.91	-3.55	22
960	372	8	0.1	16	0.00010	48	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-5.22	-5.35	-7.87	-3.15	-0.79	-4.35	-3.94	-3.58	-3.29	-4.00	-4.20	-3.11	-4.59	-2.95	-1.48	-3.52	1.98	-5.95	0.56	-4.23	-5.78	-2.86	-1.14	0.78	-4.76	22
783	372	8	0.1	16	0.00010	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-5.92	-4.49	-3.99	-3.52	-4.12	-2.41	-6.56	-4.12	-5.36	-2.41	0.60	0.81	-5.32	-2.18	-4.00	-6.85	-6.59	-4.20	-0.58	1.18	-6.69	-2.40	-7.42	-3.88	-7.26	22
588	372	8	0.1	16	0.00010	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.12	0.25	-4.36	-4.65	-27.89	-1.14	-5.36	-2.89	-4.59	-3.84	-0.70	0.32	-3.36	-1.83	-4.94	-0.10	-0.80	-3.67	-0.64	-0.04	-2.64	-1.68	0.88	-1.93	-0.01	22
1167	372	8	0.1	16	0.00001	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-5.01	-2.49	-6.19	-0.29	-0.53	-2.38	-3.28	-2.00	-1.97	-3.25	0.31	-2.25	-28.27	-0.84	-4.43	-4.04	-1.55	-2.92	0.64	1.12	-4.05	-1.05	-2.41	-1.56	-1.55	22
387	372	8	0.1	16	0.00100	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-8.15	-5.35	-8.29	-3.60	-5.70	4.40	-29.63	-43.28	-3.25	-5.20	-1.60	-0.36	-31.20	-1.28	-2.62	8.81	-3.72	-4.49	1.42	-2.05	-2.28	-0.59	-3.84	-7.10	2.54	21
780	372	8	0.1	16	0.00010	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-5.53	-5.53	-4.44	-1.90	-6.13	-0.39	-3.19	-3.04	-4.57	-6.25	-2.76	1.19	-1.57	-2.38	-0.89	-3.21	-1.59	-3.86	-0.15	0.30	0.13	-2.44	-1.53	-1.47	0.06	21
643	372	8	0.1	32	0.00010	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-1.89	-3.83	-3.01	-3.16	-2.54	-29.32	-29.46	-0.26	4.40	-2.67	1.71	0.41	-3.10	-2.41	-2.91	-30.30	-0.20	-1.31	-0.27	1.63	-1.79	-2.52	0.96	2.12	-1.43	19
1219	372	8	0.1	32	0.00001	24	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.38	-29.08	-0.56	-1.29	0.25	-31.44	-28.32	-1.92	-31.24	-1.17	-35.76	-31.22	-5.78	-0.75	-0.21	-32.21	1.43	-1.56	0.44	1.82	-2.21	1.05	1.81	-2.20	2.31	18
259	372	8	0.1	32	0.00100	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-3.33	-2.85	-0.42	-0.75	-1.37	-30.72	-30.45	-1.58	-31.67	-2.61	-34.43	2.85	0.17	-6.29	-1.23	28.70	-3.10	1.60	0.23	-30.01	-1.37	-0.14	1.62	-5.55	2.31	18
1027	372	8	0.1	32	0.00010	48	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-4.43	-1.30	-28.24	0.33	-4.42	-31.32	-28.90	-0.71	-2.31	4.40	-39.95	-32.21	-1.48	-37.44	1.98	-6.88	-0.27	1.99	-32.24	3.31	-0.54	1.94	-1.66	7.52	-0.13	18
1024	372	8	0.1	32	0.00010	48	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-3.20	-0.61	-9.04	-2.05	-1.71	3.18	-30.63	-1.55	-0.13	-1.34	1.28	-6.07	-31.02	-30.62	-6.13	-1.35	-0.67	-32.85	0.54	1.34	-4.43	-3.25	2.86	3.95	0.45	18
832	372	8	0.1	32	0.00010	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-2.63	-2.49	-29.42	-2.98	-0.74	-1.16	-23.92	-2.63	3.25	-2.12	-39.63	0.13	-2.96	-1.00	-2.09	-4.01	-1.62	-0.48	2.35	-32.96	-3.76	0.80	0.85	2.04	1.49	18
847	372	8	0.1	32	0.00010	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.53	-29.23	-0.88	-1.52	-2.72	1.60	-29.01	0.94	-6.77	-1.09	-31.47	2.14	-31.65	-1.39	-0.06	-2.80	-0.45	-2.79	-33.01	2.14	0.76	0.93	0.03	-0.28	1.37	17
79	372	8	0.1	32	0.00100	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.37	-0.39	-30.29	-1.24	-1.07	-33.10	-28.74	-1.24	-32.40	-1.80	-32.21	3.03	2.48	-1.35	4.40	-0.22	-0.11	-0.92	1.51	2.99	-3.13	-0.02	3.25	0.15	2.95	17
1423	372	8	0.1	32	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-1.61	-1.23	-0.66	-30.84	0.70	0.01	-4.68	-0.13	-1.08	-1.77	-33.91	1.12	-32.40	0.13	-1.01	-1.73	-0.78	-0.49	-32.83	2.99	-1.48	0.90	-0.86	0.65	3.05	17
1420	372	8	0.1	32	0.00001	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-8.05	-1.90	-5.26	-2.38	-3.19	-31.76	-6.84	-3.95	-2.54	-1.56	2.74	1.08	-31.07	-31.50	-1.72	-0.56	-1.42	-1.71	1.93	1.50	0.33	0.95	1.24	-7.96	2.55	17
640	372	8	0.1	32	0.00010	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-28.79	-2.48	-4.88	-1.28	-30.50	0.48	-2.81	0.02	-1.79	-31.67	-32.09	1.55	-33.31	0.76	0.85	-0.83	-0.17	-3.17	2.27	1.45	-1.42	-0.45	3.44	-2.16	-1.38	17
911	372	8	0.1	64	0.00010	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.58	0.98	-30.15	-29.88	-30.29	2.43	-30.80	-0.81	-30.36	1.27	-31.20	-32.47	-31.00	-33.18	3.14	-33.06	-1.49	-33.02	5.62	1.58	-1.23	1.28	3.83	-0.47	2.05	16
704	372	8	0.1	64	0.00010	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-4.00	-0.66	-1.52	-29.08	-1.33	0.93	-6.97	0.76	-2.85	-2.87	-30.45	2.34	-40.29	-0.67	0.45	-32.28	0.83	-31.02	5.43	-32.92	-4.77	3.57	2.52	3.83	-3.66	16
256	372	8	0.1	32	0.00100	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.03	-2.99	-28.71	-0.80	-3.45	0.73	-30.19	-0.20	-2.88	-29.32	2.40	1.54	-29.06	-0.69	-3.82	-33.22	1.63	-5.78	0.71	-33.11	3.07	2.17	2.05	6.32	2.19	15
1231	372	8	0.1	32	0.00001	24	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.16	-2.27	-1.93	-0.67	-1.57	-31.48	-0.83	1.33	-2.81	-0.65	-31.88	0.93	-33.36	3.23	-3.00	-31.76	0.09	-0.07	1.21	2.53	2.82	-0.88	0.20	0.66	1.82	15
652	372	8	0.1	32	0.00010	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.24	-2.55	-1.20	0.02	-0.62	-32.70	-29.97	1.98	-29.39	-0.08	-31.03	2.19	-31.31	1.08	-1.68	0.69	-0.97	2.03	2.19	1.26	2.60	-1.04	3.24	1.43	-0.06	14
1103	372	8	0.1	64	0.00010	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-1.15	-30.23	-2.43	-0.45	-0.55	-33.33	-28.14	0.55	-0.55	-30.60	3.80	1.74	-28.29	-0.34	-0.03	-31.34	1.72	1.75	-33.48	3.62	6.06	4.68	3.82	2.61	3.01	14
463	372	8	0.1	32	0.00100	48	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-31.14	-1.83	-0.58	0.33	-1.19	-0.37	-30.53	-1.89	-30.86	-2.12	0.72	3.12	4.40	0.50	-1.20	-32.42	-2.75	0.07	3.12	2.67	0.35	5.06	-28.06	1.24	1.51	13
1100	372	8	0.1	64	0.00010	48	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.76	-2.58	-1.47	-1.29	-2.84	-31.86	-30.48	-0.58	0.62	1.33	2.60	2.37	-32.93	1.11	0.44	-32.26	0.18	1.67	-32.74	-34.31	3.55	2.37	5.35	2.90	-3.04	13
140	372	8	0.1	64	0.00100	24	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.47	0.10	0.16	-0.80	-27.61	-30.62	-29.60	-1.48	0.42	1.18	-39.16	2.72	-31.67	0.88	1.21	-40.48	-33.67	1.94	3.00	2.85	4.76	-0.19	-30.35	2.33	5.19	12
320	372	8	0.1	64	0.00100	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.20	-3.36	-0.55	0.89	-1.69	3.02	-29.97	0.41	-32.07	-0.08	1.96	4.40	-32.59	1.20	-0.36	-31.61	0.83	-32.95	0.81	3.04	2.86	-2.09	2.10	1.88	4.26	12
908	372	8	0.1	64	0.00010	36	1.000000e-08	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-32.53	-30.08	-2.96	0.29	-30.02	-31.73	-33.54	2.45	-0.22	-30.20	-35.89	-36.72	-29.53	3.79	2.25	-31.28	3.44	0.45	4.10	4.21	1.51	2.77	1.94	4.58	5.60	12
1472	372	8	0.1	64	0.00001	36	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.59	1.31	-29.61	-0.52	-0.68	0.32	2.23	0.84	-32.61	-1.99	2.40	3.62	-34.20	2.36	1.69	1.79	1.17	-32.73	1.80	-39.64	-4.36	-6.23	2.73	1.36	1.63	11
1487	372	8	0.1	64	0.00001	36	1.000000e-08	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-27.72	-2.91	-30.54	0.13	-29.81	3.23	-29.92	2.87	-31.57	-1.40	-32.52	2.08	1.43	0.52	1.40	19.64	-0.70	-31.12	1.63	2.57	-29.40	2.35	4.28	3.58	3.80	11
1280	372	8	0.1	64	0.00001	24	0.000000e+00	0.000000e+00	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-29.71	0.22	-0.00	-30.23	-28.62	-32.30	-29.14	1.42	-1.24	-1.70	-0.12	2.98	-31.67	1.52	1.45	-32.41	2.21	2.58	2.31	2.79	2.92	6.81	2.01	2.75	1.83	10
899	372	8	0.1	64	0.00010	36	0.000000e+00	1.000000e-08	0	0	-0.0	0	-0.0	0	-0.0	-0.0	-30.52	-0.08	-30.17	-28.48	0.31	1.84	-28.25	0.77	1.34	0.55	2.84	3.24	-32.40	1.53	0.50	-32.85	4.30	-33.41	3.35	-34.23	2.58	4.58	1.39	3.43	4.25	9

Model performance on valid data

Average & Std Deviation of the Significant rows:

	mean	std
mAP_valid_zero_13	75.967778	14.634333
mAP_valid_zero_7	75.602667	13.243779
mAP_valid_zero	74.252222	11.498589
mAP_valid_zero_11	70.907111	14.550956
mAP_valid_zero_16	70.425556	14.419383
mAP_valid_zero_6	68.959111	14.515147
mAP_valid_zero_9	68.352667	12.177099
mAP_valid_zero_18	67.892444	11.195342
mAP_valid_zero_3	67.790667	9.876929
mAP_valid_zero_5	67.216889	9.455073
mAP_valid_zero_4	66.162889	8.540801
mAP_valid_zero_2	65.894222	7.496300
mAP_valid_zero_10	65.491333	8.828516
mAP_valid_zero_14	65.043111	9.350688
mAP_valid_zero_20	64.873778	12.273670
mAP_valid_zero_21	63.788222	4.938599
mAP_valid_zero_19	63.638889	11.703670
mAP_valid_zero_23	63.205111	6.306356
mAP_valid_zero_17	62.676889	5.343262
mAP_valid_zero_22	62.537111	3.402247
mAP_valid_zero_8	62.498667	3.567790
mAP_valid_zero_12	62.351556	9.548895
mAP_valid_zero_25	62.211111	4.397973
mAP_valid_zero_15	62.002444	3.491375
mAP_valid_zero_26	61.901111	3.980599

Model performance on Test data

We then apply the hyperparameters to the test set and average the results.

####################################################
# Use the hyperparameters to the test data.
####################################################
optimal_params = dt_valid.loc[dt_valid_sub_mw1.index].iloc[:, :6].reset_index()
optimal_params.drop(['index'], axis=1, inplace = True)

#rotation invariance test data with optimal parameters
test_data_1 = pd.merge(optimal_params, dt_test, how='left')
test_data_1_sub = test_data_1[list(test_data_1.columns[test_data_1.columns.str.startswith('mAP_test')])]
test_data_1['average_map'] = test_data_1_sub.apply(np.mean, axis=1)

test_data_1.sort_values(by=['average_map'], ascending= False,inplace=True)

	mean	std
mAP_test_zero_21	61.795056	2.670702
mAP_test_zero_26	61.448556	1.796655
mAP_test_zero_25	61.323111	2.849762
mAP_test_zero_23	61.305333	3.059522
mAP_test_zero_22	61.210056	2.535423
mAP_test_zero_18	60.359333	4.610879
mAP_test_zero_15	60.297889	1.840798
mAP_test_zero_14	60.259056	4.319369
mAP_test_zero_4	60.229444	1.728561
mAP_test_zero_2	60.215222	2.214090
mAP_test_zero_8	60.044722	4.867475
mAP_test_zero_10	59.973889	3.985827
mAP_test_zero_19	59.941278	6.292059
average_map	59.808222	0.963510
mAP_test_zero_17	59.781222	2.591528
mAP_test_zero_5	59.730944	2.200121
mAP_test_zero_20	59.698278	4.701437
mAP_test_zero_12	59.669333	2.440527
mAP_test_zero_3	59.146278	3.201553
mAP_test_zero_11	59.037444	3.876609
mAP_test_zero_6	58.992722	5.486129
mAP_test_zero_16	58.885889	4.457514
mAP_test_zero_9	58.767722	4.992287
mAP_test_zero_13	58.290500	5.490554
mAP_test_zero	58.211833	4.837632
mAP_test_zero_7	56.590444	6.297457

Summary using radar plot

res1_valid['id'] = res1_valid.index.to_series().apply(extract_number)
res1_test['id'] = res1_test.index.to_series().apply(extract_number)



res_comb = pd.concat([res1_valid,res1_test])
index_series = res_comb.index.to_series()

res_comb['type'] = np.where(
    index_series.str.contains('valid', case=False), 'valid',
    np.where(index_series.str.contains('test', case=False), 'test', 'unknown')
)
res_comb = res_comb.query("type !='unknown'")

res_comb =  res_comb.reset_index(drop=True)


res1_test = res1_test.sort_values(by=['id']).reset_index().query("index !='average_map'")

data_range1 = np.array(list(res1_valid['mean']) + list(res1_test['mean']))

categories = [str(i) for i in range(1,26)]

fig = go.Figure()

# Valid
fig.add_trace(go.Scatterpolar(
       r=list(res1_test['mean']),
      theta=categories,
      #fill='toself',
      name='Test'
))

# Test
fig.add_trace(go.Scatterpolar(
      r=list(res1_valid['mean']),
      theta=categories,
      #fill='toself',
      name='Valid'
))

# Customization of chart
fig.update_layout(
  title={
      'text': "T-Test",
        'xanchor': 'center',
      'yanchor': 'top'
  },
  polar=dict(
    radialaxis=dict(
      visible=True,
      range=[round(data_range1.min()), round(data_range1.max()) + 1]
    )),
  showlegend=True
)

fig.show()




##############


res2_valid['id'] = res2_valid.index.to_series().apply(extract_number)
res2_test['id'] = res2_test.index.to_series().apply(extract_number)


res_comb = pd.concat([res2_valid,res2_test])
index_series = res_comb.index.to_series()

res_comb['type'] = np.where(
    index_series.str.contains('valid', case=False), 'valid',
    np.where(index_series.str.contains('test', case=False), 'test', 'unknown')
)
res_comb = res_comb.query("type !='unknown'")

res_comb =  res_comb.reset_index(drop=True)


res2_test = res2_test.sort_values(by=['id']).reset_index().query("index !='average_map'")

data_range2 = np.array(list(res2_valid['mean']) + list(res2_test['mean']))

categories = [str(i) for i in range(1,26)]
fig = go.Figure()

# Valid
fig.add_trace(go.Scatterpolar(
       r=list(res2_test['mean']),
      theta=categories,
      #fill='toself',
      name='Test'
))

# Test
fig.add_trace(go.Scatterpolar(
      r=list(res2_valid['mean']),
      theta=categories,
      #fill='toself',
      name='Valid'
))

# Customization of chart
fig.update_layout(
  title={
      'text': "MU-Test",
         'xanchor': 'center',
      'yanchor': 'top'
  },
  polar=dict(
    radialaxis=dict(
      visible=True,
      range=[round(data_range2.min()), round(data_range2.max()) + 1]
    )),
  showlegend=True
)

fig.show()

best_valid_bit_size_8 = round(res1_valid['mean'][0])
id_best_valid = res1_valid['id'][0]
best_test_bit_size_8 = list(round(res1_test.query('id == @id_best_valid')['mean'],2))[0]

best_valid_bit_size_8_mw = round(res2_valid['mean'][0])
id_best_valid = res2_valid['id'][0]
best_test_bit_size_8_mw = list(round(res2_test.query('id == @id_best_valid')['mean'],2))[0]

Comparisons of bit size results

T-test curves

MU-test curves